[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:33 dist-test-slave-9jjr krb5kdc[28118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:33 dist-test-slave-9jjr krb5kdc[28118](info): set up 2 sockets
Aug 11 18:49:33 dist-test-slave-9jjr krb5kdc[28118](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:49:34.881560 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:34 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:35.027338 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43431
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:43431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:35.166985 28132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:35.167300 28132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:35.167392 28132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:35.171732 28132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:35.171818 28132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:35.171838 28132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:35.171855 28132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:35.171871 28132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:35.177909 28132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:43431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:35.179577 28132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:35.181044 28132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:35.195093 28139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:35.195111 28143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:35.197105 28132 server_base.cc:1034] running on GCE node
W20260811 18:49:35.197167 28140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:35.198521 28132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:35.200850 28132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:35.202436 28132 hybrid_clock.cc:648] HybridClock initialized: now 1786474175202352 us; error 114 us; skew 500 ppm
Aug 11 18:49:35 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474175, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:35.209909 28132 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:35.211833 28132 webserver.cc:460] Webserver started at http://127.27.115.254:38349/ using document root <none> and password file <none>
I20260811 18:49:35.213017 28132 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:35.213150 28132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:35.213634 28132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:35.216773 28132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "2e2fda82a45e47b3ac209f72c0b692f2"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "5b083445edc972ac9dc067222bfb9268"
server_key_iv: "bcb20ab0ea68bf1fc71030cc2fdce209"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.217653 28132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "2e2fda82a45e47b3ac209f72c0b692f2"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "5b083445edc972ac9dc067222bfb9268"
server_key_iv: "bcb20ab0ea68bf1fc71030cc2fdce209"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.224390 28132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 18:49:35.228912 28151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.231382 28132 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:49:35.231757 28132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "2e2fda82a45e47b3ac209f72c0b692f2"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "5b083445edc972ac9dc067222bfb9268"
server_key_iv: "bcb20ab0ea68bf1fc71030cc2fdce209"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.231951 28132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:35.272632 28132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:35.276919 28132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:35.277217 28132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:35.292415 28132 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:43431
I20260811 18:49:35.292586 28214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:43431 every 8 connection(s)
I20260811 18:49:35.294107 28132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:35.299149 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28132
I20260811 18:49:35.299472 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:35.301210 28215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:35.302059 28111 external_mini_cluster.cc:1428] Setting key 71221e6fc7e35886b7ea4d0801d1b842
I20260811 18:49:35.315305 28215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2: Bootstrap starting.
I20260811 18:49:35.322117 28215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:35.324795 28215 log.cc:824] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:35.329499 28215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2: No bootstrap required, opened a new log
Aug 11 18:49:35 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:35.338871 28215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } }
I20260811 18:49:35.339298 28215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:35.339428 28215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e2fda82a45e47b3ac209f72c0b692f2, State: Initialized, Role: FOLLOWER
I20260811 18:49:35.340432 28215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } }
I20260811 18:49:35.340718 28215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:35.340865 28215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:35.341015 28215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:35.342535 28215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } }
I20260811 18:49:35.343326 28215 leader_election.cc:302] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e2fda82a45e47b3ac209f72c0b692f2; no voters:
I20260811 18:49:35.343819 28215 leader_election.cc:288] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:35.344010 28221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:35.344274 28221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [term 1 LEADER]: Becoming Leader. State: Replica: 2e2fda82a45e47b3ac209f72c0b692f2, State: Running, Role: LEADER
I20260811 18:49:35.344859 28221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } }
I20260811 18:49:35.345924 28215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:35.347208 28221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e2fda82a45e47b3ac209f72c0b692f2. Latest consensus state: current_term: 1 leader_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } } }
I20260811 18:49:35.347363 28221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:35.348465 28228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:35.349900 28222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2fda82a45e47b3ac209f72c0b692f2" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43431 } } }
I20260811 18:49:35.350050 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:35.311151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.27.115.254:43431)
0811 18:49:35.311931 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:35.311982 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:35.312053 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:49:35.319847 (+ 7794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:35.319870 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:35.320424 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:35.320784 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:35.326251 (+ 5467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.327928 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.330556 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.331210 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.331371 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:35.339010 (+ 7639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:35.339060 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:35.339077 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:35.339127 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:35.341906 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:35.342740 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:35.342748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:35.342756 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:35.342910 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:35.343372 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:35.343376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:35.343378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:35.344206 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:35.346939 (+ 2733us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:35.347154 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:49:35.347553 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":299,"threads_started":1}
I20260811 18:49:35.350343 28222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:35.356639 28228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:35.368590 28228 catalog_manager.cc:1347] Generated new cluster ID: 7000a31399364e67a626c856242b7b61
I20260811 18:49:35.368729 28228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:35.379559 28228 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:35.381278 28228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:35.390854 28228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e2fda82a45e47b3ac209f72c0b692f2: Generated new TSK 0
I20260811 18:49:35.392007 28228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:35.434762 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:35.609376 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:35.609694 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:35.609750 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:35.614279 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:35.614352 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:35.614462 28244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:35.620780 28244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:35.622189 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:35.623446 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:49:35.637748 28244 server_base.cc:1034] running on GCE node
W20260811 18:49:35.637676 28250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:35.637677 28251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:35.637679 28253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:35.638528 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:35.639295 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:35.640522 28244 hybrid_clock.cc:648] HybridClock initialized: now 1786474175640490 us; error 44 us; skew 500 ppm
Aug 11 18:49:35 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474175, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:35.644359 28244 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:35.645279 28244 webserver.cc:460] Webserver started at http://127.27.115.193:34813/ using document root <none> and password file <none>
I20260811 18:49:35.645833 28244 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:35.645893 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:35.646064 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:35.648488 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "d38f2b6a6e224c9e8c1c8cff33ce7e04"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "601fada9e5a43734c4604712a82542a5"
server_key_iv: "f02368658dc486652af94b52f1d9d1cb"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.649235 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "d38f2b6a6e224c9e8c1c8cff33ce7e04"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "601fada9e5a43734c4604712a82542a5"
server_key_iv: "f02368658dc486652af94b52f1d9d1cb"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.653630 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:49:35.656661 28260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.657998 28244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:49:35.658159 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "d38f2b6a6e224c9e8c1c8cff33ce7e04"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "601fada9e5a43734c4604712a82542a5"
server_key_iv: "f02368658dc486652af94b52f1d9d1cb"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.658289 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:35.672698 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:35.676673 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:35.676919 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:35.677827 28244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:35.679090 28244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:35.679162 28244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.679234 28244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:35.679265 28244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.701484 28244 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:41123
I20260811 18:49:35.701578 28395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:41123 every 8 connection(s)
I20260811 18:49:35.702667 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:35.705569 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28244
I20260811 18:49:35.705694 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:35.705973 28111 external_mini_cluster.cc:1428] Setting key 4a358783cf8e1d1eee4a6d38820f688f
Aug 11 18:49:35 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474175, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:49:35.722497 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:35.705235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:51329 (local address 127.27.115.254:43431)
0811 18:49:35.705431 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:35.705436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:35.706911 (+ 1475us) server_negotiation.cc:407] Connection header received
0811 18:49:35.708169 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:35.708172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:35.708228 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:35.708334 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:35.710401 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.710993 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.712836 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.713370 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.713514 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:35.717066 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:35.717072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:35.717075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:35.717114 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:35.719451 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:35.720368 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:35.720372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:35.720373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:35.720468 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:35.720928 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:35.720932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:35.720933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:35.721182 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:35.721282 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:35.721970 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 18:49:35.722235 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:35.723719 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:35.705616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43431 (local address 127.27.115.193:51329)
0811 18:49:35.706732 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:35.706792 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:35.707944 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:35.708524 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:35.708532 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:35.709083 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:35.710186 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:35.710213 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.711210 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.711215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:35.712559 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:35.712572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:35.713554 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:35.713557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:35.713776 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:35.714817 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:35.714837 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:35.716813 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:35.719756 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:35.719772 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:35.719790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:35.720142 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:35.720554 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:35.720557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:35.720558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:35.720763 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:35.721337 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:35.721344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:35.721662 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 18:49:35.721854 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:49:35.722413 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":223,"threads_started":1}
I20260811 18:49:35.725783 28396 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43431
I20260811 18:49:35.726092 28396 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:35.726718 28396 heartbeater.cc:511] Master 127.27.115.254:43431 requested a full tablet report, sending...
I20260811 18:49:35.729148 28173 ts_manager.cc:194] Registered new tserver with Master: d38f2b6a6e224c9e8c1c8cff33ce7e04 (127.27.115.193:41123)
I20260811 18:49:35.730787 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:51329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:35.742046 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:35.902921 28402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:35.903285 28402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:35.903339 28402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:35.907873 28402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:35.907972 28402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:35.908063 28402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:35.914283 28402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:35.915557 28402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:35.916719 28402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:35.929728 28412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:35.929644 28410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:35.929672 28409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:35.930943 28402 server_base.cc:1034] running on GCE node
I20260811 18:49:35.931435 28402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:35.932199 28402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:35.933488 28402 hybrid_clock.cc:648] HybridClock initialized: now 1786474175933469 us; error 32 us; skew 500 ppm
Aug 11 18:49:35 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474175, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:35.937686 28402 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:35.938844 28402 webserver.cc:460] Webserver started at http://127.27.115.194:42189/ using document root <none> and password file <none>
I20260811 18:49:35.939512 28402 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:35.939584 28402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:35.939828 28402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:35.942548 28402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "c1e667ecd8e3457395e020b5d6da4eba"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "20b58b25140272f35eaa50dd28f89e31"
server_key_iv: "9abf3229135c2d5b5ad3e958c99671d8"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.943332 28402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "c1e667ecd8e3457395e020b5d6da4eba"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "20b58b25140272f35eaa50dd28f89e31"
server_key_iv: "9abf3229135c2d5b5ad3e958c99671d8"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.948858 28402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:49:35.951924 28419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.953368 28402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:49:35.953509 28402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "c1e667ecd8e3457395e020b5d6da4eba"
format_stamp: "Formatted at 2026-08-11 18:49:35 on dist-test-slave-9jjr"
server_key: "20b58b25140272f35eaa50dd28f89e31"
server_key_iv: "9abf3229135c2d5b5ad3e958c99671d8"
server_key_version: "encryptionkey@0"
I20260811 18:49:35.953646 28402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:35.980024 28402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:35.983974 28402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:35.984203 28402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:35.984947 28402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:35.985888 28402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:35.985937 28402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:35.985980 28402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:35.985996 28402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:36.004879 28402 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:46877
I20260811 18:49:36.004906 28550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:46877 every 8 connection(s)
I20260811 18:49:36.005959 28402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:36.012334 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28402
I20260811 18:49:36.012476 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:36.012758 28111 external_mini_cluster.cc:1428] Setting key 0a9fa10f3e2858d974807af702d2b41b
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474175, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:36.024312 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.008154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:47225 (local address 127.27.115.254:43431)
0811 18:49:36.008306 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:36.008309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:36.009734 (+ 1425us) server_negotiation.cc:407] Connection header received
0811 18:49:36.011117 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:36.011120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:36.011189 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:36.011305 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:36.013045 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.013703 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.015296 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.015523 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.015643 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.019288 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:36.019292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:36.019294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:36.019323 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:36.021255 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.022183 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.022191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.022193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.022309 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.022883 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.022885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.022887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.023106 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:36.023193 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:36.024055 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 18:49:36.024175 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:49:36.025959 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.008498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43431 (local address 127.27.115.194:47225)
0811 18:49:36.009524 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:36.009584 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:36.010866 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:36.011506 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:36.011514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:36.012131 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:36.012808 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:36.012826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.013849 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.013854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:36.015143 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:36.015154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.015694 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.015698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:36.015950 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.016729 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:36.016747 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:36.019001 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:36.021423 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:36.021434 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:36.021461 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:36.021943 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:36.022424 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:36.022430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:36.022434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:36.022741 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:36.023252 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:36.023273 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:36.023726 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 18:49:36.024005 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 18:49:36.024529 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":226,"threads_started":1}
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:49:36.027601 28551 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43431
I20260811 18:49:36.027979 28551 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:36.028606 28551 heartbeater.cc:511] Master 127.27.115.254:43431 requested a full tablet report, sending...
I20260811 18:49:36.030048 28173 ts_manager.cc:194] Registered new tserver with Master: c1e667ecd8e3457395e020b5d6da4eba (127.27.115.194:46877)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:49:36.030740 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:47225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:36.051342 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:36.213258 28557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:36.213650 28557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:36.213737 28557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:36.218427 28557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:36.218546 28557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:36.218652 28557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:36.225277 28557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:36.226735 28557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:36.227957 28557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:36.240250 28565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:36.240252 28563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:36.240310 28562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:36.240927 28557 server_base.cc:1034] running on GCE node
I20260811 18:49:36.241470 28557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:36.242378 28557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:36.243559 28557 hybrid_clock.cc:648] HybridClock initialized: now 1786474176243513 us; error 65 us; skew 500 ppm
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:36.246764 28557 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:36.247871 28557 webserver.cc:460] Webserver started at http://127.27.115.195:46777/ using document root <none> and password file <none>
I20260811 18:49:36.248493 28557 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:36.248615 28557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:36.248880 28557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:36.251303 28557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "e10ed65c2f9d4a2687d951d60371219b"
format_stamp: "Formatted at 2026-08-11 18:49:36 on dist-test-slave-9jjr"
server_key: "abf010e541ffb823ffe44799f64dba3d"
server_key_iv: "93da2253d76690bb50a8a28dee18a4ca"
server_key_version: "encryptionkey@0"
I20260811 18:49:36.251930 28557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "e10ed65c2f9d4a2687d951d60371219b"
format_stamp: "Formatted at 2026-08-11 18:49:36 on dist-test-slave-9jjr"
server_key: "abf010e541ffb823ffe44799f64dba3d"
server_key_iv: "93da2253d76690bb50a8a28dee18a4ca"
server_key_version: "encryptionkey@0"
I20260811 18:49:36.256537 28557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:49:36.259615 28573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:36.261313 28557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:49:36.261466 28557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "e10ed65c2f9d4a2687d951d60371219b"
format_stamp: "Formatted at 2026-08-11 18:49:36 on dist-test-slave-9jjr"
server_key: "abf010e541ffb823ffe44799f64dba3d"
server_key_iv: "93da2253d76690bb50a8a28dee18a4ca"
server_key_version: "encryptionkey@0"
I20260811 18:49:36.261642 28557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:36.276340 28557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:36.280507 28557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:36.280787 28557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:36.281627 28557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:36.282864 28557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:36.282948 28557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:36.283030 28557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:36.283099 28557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:36.309378 28557 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:46319
I20260811 18:49:36.309551 28700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:46319 every 8 connection(s)
I20260811 18:49:36.310536 28557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:36.311858 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28557
I20260811 18:49:36.311988 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:36.312274 28111 external_mini_cluster.cc:1428] Setting key 81da3acf6bd59209d5ce6db3dc679017
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:36.329099 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.313015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:38795 (local address 127.27.115.254:43431)
0811 18:49:36.313218 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:36.313223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:36.314797 (+ 1574us) server_negotiation.cc:407] Connection header received
0811 18:49:36.316236 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:36.316238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:36.316299 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:36.316381 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:36.318264 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.318839 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.320352 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.320635 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.320746 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.324345 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:36.324349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:36.324351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:36.324386 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:36.326025 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.326984 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.326989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.326990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.327102 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.327612 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.327615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.327616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.327916 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:36.328008 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:36.328636 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 18:49:36.328766 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 18:49:36.330020 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.313501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43431 (local address 127.27.115.195:38795)
0811 18:49:36.314601 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:36.314670 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:36.315952 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:36.316584 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:36.316593 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:36.317269 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:36.318029 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:36.318048 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.318994 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.318997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:36.320181 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:36.320194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.320863 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.320866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:36.321114 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.322026 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:36.322045 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:36.324087 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:36.326263 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:36.326269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:36.326287 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:36.326778 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:36.327229 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:36.327233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:36.327236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:36.327467 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:36.328045 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:36.328053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:36.328380 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 18:49:36.328563 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 18:49:36.328897 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":254,"threads_started":1}
I20260811 18:49:36.331681 28702 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43431
I20260811 18:49:36.331991 28702 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:36.332587 28702 heartbeater.cc:511] Master 127.27.115.254:43431 requested a full tablet report, sending...
I20260811 18:49:36.333847 28173 ts_manager.cc:194] Registered new tserver with Master: e10ed65c2f9d4a2687d951d60371219b (127.27.115.195:46319)
I20260811 18:49:36.334561 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:38795
I20260811 18:49:36.340431 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:36.374527 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.361996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.27.115.193:41123)
0811 18:49:36.362418 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:36.362428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:36.362572 (+ 144us) server_negotiation.cc:407] Connection header received
0811 18:49:36.362763 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:36.362771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:36.362985 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:36.363178 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:36.364121 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.364935 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.366003 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.366683 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.366826 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.369888 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:36.369903 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:36.369920 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:36.369956 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:36.372258 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.372810 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.372817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.372818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.372918 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.373268 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.373271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.373273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.373602 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:36.373799 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:36.374061 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:49:36.374364 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
W20260811 18:49:36.375856 28295 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44332
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:36.408492 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:36.399003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.27.115.193:41123)
0811 18:49:36.399209 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:36.399214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:36.399362 (+ 148us) server_negotiation.cc:407] Connection header received
0811 18:49:36.399559 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:36.399562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:36.399609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:36.399702 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:36.400501 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.401055 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.401933 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:36.402264 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:36.402381 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:36.405076 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:36.405081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:36.405083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:36.405116 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:36.406618 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.407121 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.407124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.407126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.407222 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:36.407576 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:36.407578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:36.407579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:36.407792 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:36.407908 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:36.408221 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:49:36.408361 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:49:36.410210 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28244
I20260811 18:49:36.419515 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28402
I20260811 18:49:36.428447 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28557
I20260811 18:49:36.438803 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28132
2026-08-11T18:49:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2719 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28720](info): set up 2 sockets
Aug 11 18:49:36 dist-test-slave-9jjr krb5kdc[28720](info): commencing operation
krb5kdc: starting...
W20260811 18:49:37.567139 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:37 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:37.711647 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35075
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:37.834164 28733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:37.834544 28733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:37.834637 28733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:37.838874 28733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:37.838975 28733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:37.839015 28733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:37.839074 28733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:37.839110 28733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:37.844795 28733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:37.846086 28733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:37.847231 28733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:37.856695 28740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:37.856709 28742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:37.856899 28733 server_base.cc:1034] running on GCE node
W20260811 18:49:37.857118 28739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:37.857812 28733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:37.859391 28733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:37.860666 28733 hybrid_clock.cc:648] HybridClock initialized: now 1786474177860629 us; error 62 us; skew 500 ppm
Aug 11 18:49:37 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474177, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:37.864216 28733 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:37.865320 28733 webserver.cc:460] Webserver started at http://127.27.115.254:32807/ using document root <none> and password file <none>
I20260811 18:49:37.865942 28733 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:37.866044 28733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:37.866336 28733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:37.868536 28733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "5e211c0744f34fc39105ac49d936aa84"
format_stamp: "Formatted at 2026-08-11 18:49:37 on dist-test-slave-9jjr"
server_key: "a35fe88422b32533fb82ae3c0d47191f"
server_key_iv: "eafafb82c183e05c0a4ae823f4f3ffc0"
server_key_version: "encryptionkey@0"
I20260811 18:49:37.869148 28733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "5e211c0744f34fc39105ac49d936aa84"
format_stamp: "Formatted at 2026-08-11 18:49:37 on dist-test-slave-9jjr"
server_key: "a35fe88422b32533fb82ae3c0d47191f"
server_key_iv: "eafafb82c183e05c0a4ae823f4f3ffc0"
server_key_version: "encryptionkey@0"
I20260811 18:49:37.873193 28733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:49:37.875850 28749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:37.877169 28733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:49:37.877318 28733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "5e211c0744f34fc39105ac49d936aa84"
format_stamp: "Formatted at 2026-08-11 18:49:37 on dist-test-slave-9jjr"
server_key: "a35fe88422b32533fb82ae3c0d47191f"
server_key_iv: "eafafb82c183e05c0a4ae823f4f3ffc0"
server_key_version: "encryptionkey@0"
I20260811 18:49:37.877444 28733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:37.893479 28733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:37.896855 28733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:37.897300 28733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:37.905827 28733 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:35075
I20260811 18:49:37.905843 28810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:35075 every 8 connection(s)
I20260811 18:49:37.907184 28733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:37.910320 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28733
I20260811 18:49:37.910480 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:37.910753 28111 external_mini_cluster.cc:1428] Setting key 8975c2ae08990f19d1a88416276d3335
I20260811 18:49:37.910867 28811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:37.918100 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84: Bootstrap starting.
Aug 11 18:49:37 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:37.921041 28811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:37.921964 28811 log.cc:824] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:37.924896 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84: No bootstrap required, opened a new log
I20260811 18:49:37.928160 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:37.912381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.27.115.254:35075)
0811 18:49:37.912998 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:37.913017 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:37.913062 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:49:37.914015 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:37.914036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:37.914507 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:37.914860 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:37.915728 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:37.916754 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:37.917683 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:37.918285 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:37.918459 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:37.921540 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:37.921547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:37.921565 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:37.921605 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:37.924036 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:37.924620 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:37.924626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:37.924633 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:37.924792 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:37.925220 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:37.925223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:37.925225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:37.925833 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:37.926044 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:37.926518 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 18:49:37.926870 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":201,"threads_started":1}
I20260811 18:49:37.930598 28811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } }
I20260811 18:49:37.930892 28811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:37.930956 28811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e211c0744f34fc39105ac49d936aa84, State: Initialized, Role: FOLLOWER
I20260811 18:49:37.931491 28811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } }
I20260811 18:49:37.931661 28811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:37.931728 28811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:37.931831 28811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:37.932926 28811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } }
I20260811 18:49:37.933322 28811 leader_election.cc:302] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e211c0744f34fc39105ac49d936aa84; no voters:
I20260811 18:49:37.933697 28811 leader_election.cc:288] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:37.933926 28816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:37.934232 28816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [term 1 LEADER]: Becoming Leader. State: Replica: 5e211c0744f34fc39105ac49d936aa84, State: Running, Role: LEADER
I20260811 18:49:37.934847 28816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } }
I20260811 18:49:37.935251 28811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:37.936622 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e211c0744f34fc39105ac49d936aa84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } } }
I20260811 18:49:37.936796 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:37.937125 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e211c0744f34fc39105ac49d936aa84. Latest consensus state: current_term: 1 leader_uuid: "5e211c0744f34fc39105ac49d936aa84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e211c0744f34fc39105ac49d936aa84" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35075 } } }
I20260811 18:49:37.937229 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:37.937520 28826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:37.941820 28826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:37.948634 28826 catalog_manager.cc:1347] Generated new cluster ID: df7a25f0cb5e48fb8519d6c5aa122f06
I20260811 18:49:37.948736 28826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:37.958396 28826 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:37.959808 28826 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:37.968235 28826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e211c0744f34fc39105ac49d936aa84: Generated new TSK 0
I20260811 18:49:37.969161 28826 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:38.005918 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:38.133144 28839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:38.133455 28839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:38.133504 28839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:38.138327 28839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:38.138418 28839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:38.138486 28839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:38.144510 28839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:38.145773 28839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:38.146909 28839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:49:38.157450 28839 server_base.cc:1034] running on GCE node
W20260811 18:49:38.157366 28846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.157366 28844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.157369 28848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:38.158089 28839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:38.158849 28839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:38.160060 28839 hybrid_clock.cc:648] HybridClock initialized: now 1786474178160024 us; error 47 us; skew 500 ppm
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:38.163573 28839 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:38.164980 28839 webserver.cc:460] Webserver started at http://127.27.115.193:40735/ using document root <none> and password file <none>
I20260811 18:49:38.165900 28839 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:38.166031 28839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:38.166390 28839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:38.169505 28839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "d8c6f049938d41d2aed436b6f26a368b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "b6154bb6aff9cd657d8c6d1ec0b11723"
server_key_iv: "f75dcde57f33a9fe1202076088cad88a"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.170343 28839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "d8c6f049938d41d2aed436b6f26a368b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "b6154bb6aff9cd657d8c6d1ec0b11723"
server_key_iv: "f75dcde57f33a9fe1202076088cad88a"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.175028 28839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:49:38.178037 28858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.179608 28839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:38.179790 28839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "d8c6f049938d41d2aed436b6f26a368b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "b6154bb6aff9cd657d8c6d1ec0b11723"
server_key_iv: "f75dcde57f33a9fe1202076088cad88a"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.179931 28839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:38.201349 28839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:38.205402 28839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:38.205667 28839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:38.206511 28839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:38.207517 28839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:38.207598 28839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.207672 28839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:38.207719 28839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.219632 28839 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:34851
I20260811 18:49:38.219724 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:34851 every 8 connection(s)
I20260811 18:49:38.220750 28839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:38.225554 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28839
I20260811 18:49:38.225660 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:38.225937 28111 external_mini_cluster.cc:1428] Setting key 9c3f619c85d3e74f57a64734ea9b3d09
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:49:38.238310 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.223033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:42045 (local address 127.27.115.254:35075)
0811 18:49:38.223272 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.223276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.224472 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 18:49:38.226356 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.226360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.226425 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.226517 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.228024 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.228588 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.229816 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.230063 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.230314 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.233403 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.233407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.233409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.233439 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.235456 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.236347 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.236352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.236354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.236461 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.236939 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.236944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.236946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.237185 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.237309 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.237917 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.238086 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260811 18:49:38.239362 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.223650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35075 (local address 127.27.115.193:42045)
0811 18:49:38.224204 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.224250 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.226065 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.226735 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.226743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.227286 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:38.227862 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.227877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.228731 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.228735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.229672 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.229683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.230385 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.230390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.230610 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.231379 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:38.231406 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:38.233160 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:38.235662 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.235672 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.235747 (+ 75us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.236215 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.236590 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.236594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.236595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.236847 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.237307 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:38.237317 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:38.237647 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.237875 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.238339 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":176,"threads_started":1}
I20260811 18:49:38.241242 28988 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:38.241566 28988 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:38.242300 28988 heartbeater.cc:511] Master 127.27.115.254:35075 requested a full tablet report, sending...
I20260811 18:49:38.244293 28773 ts_manager.cc:194] Registered new tserver with Master: d8c6f049938d41d2aed436b6f26a368b (127.27.115.193:34851)
I20260811 18:49:38.246105 28773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:42045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:38.262593 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:38.387701 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:38.388000 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:38.388073 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:38.392293 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:38.392364 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:38.392426 28993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:38.398030 28993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:38.399222 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:38.400362 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:38.411386 29001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.411394 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.411402 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:38.411588 28993 server_base.cc:1034] running on GCE node
I20260811 18:49:38.412344 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:38.413067 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:38.414285 28993 hybrid_clock.cc:648] HybridClock initialized: now 1786474178414245 us; error 49 us; skew 500 ppm
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:38.417266 28993 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:38.418310 28993 webserver.cc:460] Webserver started at http://127.27.115.194:44393/ using document root <none> and password file <none>
I20260811 18:49:38.418875 28993 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:38.418943 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:38.419117 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:38.421350 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "4ff60baeb2c64fb8bb52914fa107b41b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "2b3b39882dfd43ed214cb5f9b6910e5b"
server_key_iv: "0d18cee53b72f95ecdac6e9e9163b480"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.422214 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "4ff60baeb2c64fb8bb52914fa107b41b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "2b3b39882dfd43ed214cb5f9b6910e5b"
server_key_iv: "0d18cee53b72f95ecdac6e9e9163b480"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.426731 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:49:38.430110 29009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.432229 28993 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:38.432544 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "4ff60baeb2c64fb8bb52914fa107b41b"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "2b3b39882dfd43ed214cb5f9b6910e5b"
server_key_iv: "0d18cee53b72f95ecdac6e9e9163b480"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.432708 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:38.457929 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:38.461751 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:38.462018 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:38.462980 28993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:38.463948 28993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:38.463999 28993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.464042 28993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:38.464062 28993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.474426 28993 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:41801
I20260811 18:49:38.474447 29142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:41801 every 8 connection(s)
I20260811 18:49:38.475463 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:38.481453 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 28993
I20260811 18:49:38.481557 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:38.481902 28111 external_mini_cluster.cc:1428] Setting key 011113a207d769c70b669fd39cbb2471
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:49:38.494652 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.477649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:52793 (local address 127.27.115.254:35075)
0811 18:49:38.477827 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.477832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.479003 (+ 1171us) server_negotiation.cc:407] Connection header received
0811 18:49:38.480441 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.480445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.480509 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.480604 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.482527 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.483155 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.484700 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.485184 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.485334 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.488682 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.488686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.488689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.488723 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.490942 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.491972 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.491976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.491978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.492089 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.492722 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.492726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.492728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.493018 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.493132 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.494240 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.494411 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:49:38.495563 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.478198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35075 (local address 127.27.115.194:52793)
0811 18:49:38.478811 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.478863 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.480151 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.481013 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.481022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.481559 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:38.482332 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.482353 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.483326 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.483329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.484530 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.484543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.485397 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.485402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.485636 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.486416 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:38.486436 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:38.488316 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:38.491160 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.491173 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.491202 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.491762 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.492236 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.492243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.492246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.492535 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.493170 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:38.493179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:38.493656 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.493983 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.494513 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":148,"threads_started":1}
I20260811 18:49:38.497205 29143 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35075
I20260811 18:49:38.497478 29143 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:49:38.498215 29143 heartbeater.cc:511] Master 127.27.115.254:35075 requested a full tablet report, sending...
I20260811 18:49:38.499945 28773 ts_manager.cc:194] Registered new tserver with Master: 4ff60baeb2c64fb8bb52914fa107b41b (127.27.115.194:41801)
I20260811 18:49:38.500628 28773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:52793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:38.518533 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:38.648862 29148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:38.649166 29148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:38.649215 29148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:38.653483 29148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:38.653565 29148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:38.653635 29148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:38.661826 29148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:38.663430 29148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:38.664858 29148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:38.675807 29155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.675823 29157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.675813 29153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:38.676124 29148 server_base.cc:1034] running on GCE node
I20260811 18:49:38.676867 29148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:38.677660 29148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:38.678860 29148 hybrid_clock.cc:648] HybridClock initialized: now 1786474178678839 us; error 36 us; skew 500 ppm
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:38.682071 29148 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:38.683115 29148 webserver.cc:460] Webserver started at http://127.27.115.195:33991/ using document root <none> and password file <none>
I20260811 18:49:38.683749 29148 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:38.683812 29148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:38.684032 29148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:38.686271 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "cacb0a0c03214b7833218785e1b50ce9"
server_key_iv: "6c5fe2147470beaa60fdecc30d7f8c4f"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.686941 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "cacb0a0c03214b7833218785e1b50ce9"
server_key_iv: "6c5fe2147470beaa60fdecc30d7f8c4f"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.690851 29148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:38.693482 29166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.694934 29148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:49:38.695075 29148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-9jjr"
server_key: "cacb0a0c03214b7833218785e1b50ce9"
server_key_iv: "6c5fe2147470beaa60fdecc30d7f8c4f"
server_key_version: "encryptionkey@0"
I20260811 18:49:38.695202 29148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:38.725859 29148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:38.729573 29148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:38.729848 29148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:38.730629 29148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:38.731633 29148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:38.731684 29148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.731755 29148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:38.731796 29148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.744419 29148 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:43441
I20260811 18:49:38.744534 29293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:43441 every 8 connection(s)
I20260811 18:49:38.745502 29148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:38.748116 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 29148
I20260811 18:49:38.748234 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:38.748623 28111 external_mini_cluster.cc:1428] Setting key e0e12026290b6152190badafcb9f26c3
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:38.763542 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.747649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:54765 (local address 127.27.115.254:35075)
0811 18:49:38.747806 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.747810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.749081 (+ 1271us) server_negotiation.cc:407] Connection header received
0811 18:49:38.751298 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.751301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.751364 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.751445 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.753024 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.753603 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.755133 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.755389 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.755505 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.758850 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.758855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.758857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.758896 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.760696 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.761416 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.761420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.761421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.761516 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.761953 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.761957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.761959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.762291 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.762409 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.763201 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.763375 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:49:38.764719 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.748014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35075 (local address 127.27.115.195:54765)
0811 18:49:38.748862 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.748932 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.751036 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.751658 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.751666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.752164 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:38.752828 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.752847 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.753762 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.753765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.754950 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.754963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.755582 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.755587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.755836 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.756505 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:38.756524 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:38.758575 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:38.760887 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.760903 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.760922 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.761267 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.761641 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:38.761645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:38.761646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:38.761829 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:38.762488 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:38.762498 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:38.762869 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.763081 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.763499 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":272,"threads_started":1}
I20260811 18:49:38.766283 29294 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35075
I20260811 18:49:38.766599 29294 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:38.767195 29294 heartbeater.cc:511] Master 127.27.115.254:35075 requested a full tablet report, sending...
I20260811 18:49:38.768473 28773 ts_manager.cc:194] Registered new tserver with Master: 2580a741403b4ea4a7e7ecd6a2fb8a7a (127.27.115.195:43441)
I20260811 18:49:38.769115 28773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:54765
I20260811 18:49:38.776490 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:49:38.791607 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.781432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.27.115.254:35075)
0811 18:49:38.781657 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.781663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.782055 (+ 392us) server_negotiation.cc:407] Connection header received
0811 18:49:38.782305 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.782308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.782354 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.782443 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.783386 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.784005 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.784935 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.785190 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.785308 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.787003 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.787006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.787008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.787037 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.788843 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.789891 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.789898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.789900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.790020 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.790492 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.790495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.790497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.790844 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.790968 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.791365 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.791478 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:49:38.797968 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:38.802263 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:38.822409 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.814241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:43441 (local address 127.0.0.1:50628)
0811 18:49:38.814855 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.814913 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.815053 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.816531 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.816538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.816594 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:38.816822 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.816832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818685 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.818688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.820184 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.820197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.821616 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.821621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.821834 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.821898 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.822026 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.822226 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":187,"thread_start_us":113,"threads_started":1}
I20260811 18:49:38.822862 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.814540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:41801 (local address 127.0.0.1:40596)
0811 18:49:38.815575 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.815609 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.815752 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.816806 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.816812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.816838 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:38.816977 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.816985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818681 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.818685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.820442 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.820455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.821741 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.821744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.822546 (+ 802us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.822577 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.822654 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.822730 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":851,"thread_start_us":124,"threads_started":1}
I20260811 18:49:38.823453 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.814886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.27.115.195:43441)
0811 18:49:38.815949 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.815967 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.815991 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:49:38.816092 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.816098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.816330 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.816499 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:38.817009 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818506 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.820368 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.821429 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.822942 (+ 1513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.823138 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.823196 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.823287 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":374,"threads_started":1}
I20260811 18:49:38.823442 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.815429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.27.115.194:41801)
0811 18:49:38.816021 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.816068 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.816094 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:49:38.816177 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.816183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.816406 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.816780 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:38.817109 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818496 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.820608 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.821600 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.821881 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.822091 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.823167 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.823287 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":224,"threads_started":1}
I20260811 18:49:38.823578 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.814400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.27.115.193:34851)
0811 18:49:38.816021 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.816029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.816302 (+ 273us) server_negotiation.cc:407] Connection header received
0811 18:49:38.816390 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.816398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.816586 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.816913 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:38.817348 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818185 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.820832 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.821802 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.821978 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.822186 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.823262 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.823406 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":209,"threads_started":1}
I20260811 18:49:38.824019 29310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.814241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:34851 (local address 127.0.0.1:42734)
0811 18:49:38.816175 (+ 1934us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.816189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.816289 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.817074 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.817077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.817098 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:38.817226 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.817231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.818334 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.818338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.820693 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.820701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.822271 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.822278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.822491 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.823091 (+ 600us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.823762 (+ 671us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.823855 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1872,"mutex_wait_us":1205,"thread_start_us":192,"threads_started":1}
I20260811 18:49:38.826942 28913 tablet_service.cc:1467] Processing CreateTablet for tablet bef173efc82c43e0a71c867fbf7c57f9 (DEFAULT_TABLE table=test-table [id=e62152555822470d8e62f0488426d3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:38.827019 29065 tablet_service.cc:1467] Processing CreateTablet for tablet bef173efc82c43e0a71c867fbf7c57f9 (DEFAULT_TABLE table=test-table [id=e62152555822470d8e62f0488426d3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:38.828397 28913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef173efc82c43e0a71c867fbf7c57f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:38.828907 29065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef173efc82c43e0a71c867fbf7c57f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:38.828943 29218 tablet_service.cc:1467] Processing CreateTablet for tablet bef173efc82c43e0a71c867fbf7c57f9 (DEFAULT_TABLE table=test-table [id=e62152555822470d8e62f0488426d3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:38.830426 29218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef173efc82c43e0a71c867fbf7c57f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:38.840695 29313 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Bootstrap starting.
I20260811 18:49:38.842540 29314 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Bootstrap starting.
I20260811 18:49:38.842514 29315 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Bootstrap starting.
I20260811 18:49:38.844656 29313 tablet_bootstrap.cc:654] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:38.845476 29315 tablet_bootstrap.cc:654] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:38.845635 29314 tablet_bootstrap.cc:654] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:38.847198 29314 log.cc:824] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:38.847286 29313 log.cc:824] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:38.847276 29315 log.cc:824] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:38.849931 29314 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: No bootstrap required, opened a new log
I20260811 18:49:38.850015 29313 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: No bootstrap required, opened a new log
I20260811 18:49:38.850049 29315 tablet_bootstrap.cc:492] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: No bootstrap required, opened a new log
I20260811 18:49:38.850306 29313 ts_tablet_manager.cc:1397] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 18:49:38.850306 29315 ts_tablet_manager.cc:1397] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 18:49:38.850308 29314 ts_tablet_manager.cc:1397] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260811 18:49:38.855510 29314 raft_consensus.cc:348] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.855927 29314 raft_consensus.cc:374] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:38.855999 29314 raft_consensus.cc:729] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff60baeb2c64fb8bb52914fa107b41b, State: Initialized, Role: FOLLOWER
I20260811 18:49:38.856549 29314 consensus_queue.cc:260] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.857599 29143 heartbeater.cc:503] Master 127.27.115.254:35075 was elected leader, sending a full tablet report...
I20260811 18:49:38.857441 29313 raft_consensus.cc:348] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.857808 29313 raft_consensus.cc:374] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:38.857877 29313 raft_consensus.cc:729] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2580a741403b4ea4a7e7ecd6a2fb8a7a, State: Initialized, Role: FOLLOWER
I20260811 18:49:38.858029 29314 ts_tablet_manager.cc:1428] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 18:49:38.858493 29313 consensus_queue.cc:260] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.858632 29315 raft_consensus.cc:348] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.858996 29315 raft_consensus.cc:374] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:38.859059 29315 raft_consensus.cc:729] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c6f049938d41d2aed436b6f26a368b, State: Initialized, Role: FOLLOWER
I20260811 18:49:38.859655 29313 ts_tablet_manager.cc:1428] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20260811 18:49:38.859545 29315 consensus_queue.cc:260] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.860168 29294 heartbeater.cc:503] Master 127.27.115.254:35075 was elected leader, sending a full tablet report...
I20260811 18:49:38.860791 29315 ts_tablet_manager.cc:1428] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260811 18:49:38.860996 28988 heartbeater.cc:503] Master 127.27.115.254:35075 was elected leader, sending a full tablet report...
I20260811 18:49:38.897050 29321 raft_consensus.cc:482] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:38.897325 29321 raft_consensus.cc:504] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.898876 29321 leader_election.cc:288] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2580a741403b4ea4a7e7ecd6a2fb8a7a (127.27.115.195:43441), 4ff60baeb2c64fb8bb52914fa107b41b (127.27.115.194:41801)
I20260811 18:49:38.904244 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.899355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60329 (local address 127.27.115.194:41801)
0811 18:49:38.899598 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.899603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.900038 (+ 435us) server_negotiation.cc:407] Connection header received
0811 18:49:38.900322 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.900324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.900369 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.900454 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:38.900920 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.901589 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.902868 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.903574 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.903720 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.903782 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.904020 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.904114 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260811 18:49:38.904265 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.899353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:39997 (local address 127.27.115.195:43441)
0811 18:49:38.899531 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.899536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.900069 (+ 533us) server_negotiation.cc:407] Connection header received
0811 18:49:38.900315 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.900318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.900367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.900453 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:38.900977 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.901605 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.902949 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.903738 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.903859 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.903893 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.904091 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.904163 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:49:38.904282 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.899218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:41801 (local address 127.27.115.193:60329)
0811 18:49:38.899814 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.899828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.900190 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.900467 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.900470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.900498 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:38.900790 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.900797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.901718 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.901724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.902739 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.902749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.903737 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.903741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.903866 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.903898 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.903991 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.904082 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":2,"spinlock_wait_cycles":45696,"thread_start_us":178,"threads_started":1}
I20260811 18:49:38.904366 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.899221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:43441 (local address 127.27.115.193:39997)
0811 18:49:38.899955 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:38.899968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:38.900190 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:38.900469 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:38.900471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:38.900501 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:38.900861 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.900871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.901752 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.901755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.902846 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:38.902857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.903861 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.903865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:38.903970 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.903998 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:49:38.904065 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:49:38.904113 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":106,"spinlock_wait_cycles":52992,"thread_start_us":131,"threads_started":1}
I20260811 18:49:38.905169 29241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef173efc82c43e0a71c867fbf7c57f9" candidate_uuid: "d8c6f049938d41d2aed436b6f26a368b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" is_pre_election: true
I20260811 18:49:38.905179 29088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef173efc82c43e0a71c867fbf7c57f9" candidate_uuid: "d8c6f049938d41d2aed436b6f26a368b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" is_pre_election: true
I20260811 18:49:38.905660 29241 raft_consensus.cc:2454] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c6f049938d41d2aed436b6f26a368b in term 0.
I20260811 18:49:38.905685 29088 raft_consensus.cc:2454] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c6f049938d41d2aed436b6f26a368b in term 0.
I20260811 18:49:38.906420 28860 leader_election.cc:302] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2580a741403b4ea4a7e7ecd6a2fb8a7a, 4ff60baeb2c64fb8bb52914fa107b41b, d8c6f049938d41d2aed436b6f26a368b; no voters:
I20260811 18:49:38.906734 29321 raft_consensus.cc:2781] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:38.906837 29321 raft_consensus.cc:482] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:38.906893 29321 raft_consensus.cc:3037] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:38.908093 29321 raft_consensus.cc:504] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.908496 29321 leader_election.cc:288] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [CANDIDATE]: Term 1 election: Requested vote from peers 2580a741403b4ea4a7e7ecd6a2fb8a7a (127.27.115.195:43441), 4ff60baeb2c64fb8bb52914fa107b41b (127.27.115.194:41801)
I20260811 18:49:38.908913 29088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef173efc82c43e0a71c867fbf7c57f9" candidate_uuid: "d8c6f049938d41d2aed436b6f26a368b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff60baeb2c64fb8bb52914fa107b41b"
I20260811 18:49:38.908936 29241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef173efc82c43e0a71c867fbf7c57f9" candidate_uuid: "d8c6f049938d41d2aed436b6f26a368b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a"
I20260811 18:49:38.909065 29241 raft_consensus.cc:3037] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:38.909065 29088 raft_consensus.cc:3037] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:38.910887 29241 raft_consensus.cc:2454] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c6f049938d41d2aed436b6f26a368b in term 1.
I20260811 18:49:38.910887 29088 raft_consensus.cc:2454] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c6f049938d41d2aed436b6f26a368b in term 1.
I20260811 18:49:38.911269 28862 leader_election.cc:302] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff60baeb2c64fb8bb52914fa107b41b, d8c6f049938d41d2aed436b6f26a368b; no voters:
I20260811 18:49:38.911461 29321 raft_consensus.cc:2781] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:38.911734 29321 raft_consensus.cc:686] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [term 1 LEADER]: Becoming Leader. State: Replica: d8c6f049938d41d2aed436b6f26a368b, State: Running, Role: LEADER
I20260811 18:49:38.912110 29321 consensus_queue.cc:237] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } }
I20260811 18:49:38.915721 28770 catalog_manager.cc:5495] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b reported cstate change: term changed from 0 to 1, leader changed from <none> to d8c6f049938d41d2aed436b6f26a368b (127.27.115.193). New cstate: current_term: 1 leader_uuid: "d8c6f049938d41d2aed436b6f26a368b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c6f049938d41d2aed436b6f26a368b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 34851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2580a741403b4ea4a7e7ecd6a2fb8a7a" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 43441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ff60baeb2c64fb8bb52914fa107b41b" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 41801 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:38.933290 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.923420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.27.115.193:34851)
0811 18:49:38.923595 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.923605 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.923759 (+ 154us) server_negotiation.cc:407] Connection header received
0811 18:49:38.923871 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.923873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.923915 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.923992 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.924753 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.925262 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.926312 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.926518 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.926597 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.929316 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.929336 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.929357 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.929391 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.931582 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.932049 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.932054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.932055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.932143 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.932393 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.932395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.932396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.932677 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.932834 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.933027 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.933159 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:38.964445 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.955435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.27.115.193:34851)
0811 18:49:38.955652 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.955656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.955710 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:49:38.955852 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.955854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.955902 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.955978 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.956650 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.957194 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.958029 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.958372 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.958462 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.961175 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.961179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.961181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.961211 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.962654 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.963077 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.963080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.963081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.963181 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.963431 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.963434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.963435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.963624 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.963830 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.964119 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.964275 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 18:49:38.972493 28989 tablet.cc:2369] T bef173efc82c43e0a71c867fbf7c57f9 P d8c6f049938d41d2aed436b6f26a368b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:38.976890 29144 tablet.cc:2369] T bef173efc82c43e0a71c867fbf7c57f9 P 4ff60baeb2c64fb8bb52914fa107b41b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:49:38 dist-test-slave-9jjr krb5kdc[28720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:38.995466 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.985809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.27.115.193:34851)
0811 18:49:38.985974 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.985978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.986186 (+ 208us) server_negotiation.cc:407] Connection header received
0811 18:49:38.986383 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.986385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.986434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.986520 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.987281 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.987802 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.988786 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.989007 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.989094 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.992183 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.992187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.992189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.992227 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.993607 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.994062 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.994065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.994066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.994238 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.994517 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.994519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.994521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.994810 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.994986 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.995213 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.995332 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 18:49:38.996826 29295 tablet.cc:2369] T bef173efc82c43e0a71c867fbf7c57f9 P 2580a741403b4ea4a7e7ecd6a2fb8a7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:38.997815 28890 tablet_service.cc:2908] Rejecting scan request for tablet bef173efc82c43e0a71c867fbf7c57f9: Uninitialized: clean time has not yet been initialized
I20260811 18:49:39.001068 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28839
I20260811 18:49:39.011991 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28993
I20260811 18:49:39.025069 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 29148
I20260811 18:49:39.035331 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 28733
2026-08-11T18:49:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2598 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:39 dist-test-slave-9jjr krb5kdc[29341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:39 dist-test-slave-9jjr krb5kdc[29341](info): set up 2 sockets
Aug 11 18:49:39 dist-test-slave-9jjr krb5kdc[29341](info): commencing operation
krb5kdc: starting...
W20260811 18:49:40.204303 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:40 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:40.346434 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42613
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:42613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:40.468086 29355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:40.468485 29355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:40.468570 29355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:40.472774 29355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:40.472879 29355 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:49:40.472913 29355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:40.472955 29355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:40.472993 29355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:40.473049 29355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:40.478745 29355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:42613
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:40.480223 29355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:40.481431 29355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:40.491191 29363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.491199 29366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.491189 29364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:40.491667 29355 server_base.cc:1034] running on GCE node
I20260811 18:49:40.492417 29355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:40.493778 29355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:40.494983 29355 hybrid_clock.cc:648] HybridClock initialized: now 1786474180494957 us; error 42 us; skew 500 ppm
Aug 11 18:49:40 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:40.502713 29355 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:40.503963 29355 webserver.cc:460] Webserver started at http://127.27.115.254:40441/ using document root <none> and password file <none>
I20260811 18:49:40.504581 29355 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:40.504678 29355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:40.504918 29355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:40.507191 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "370eb7b725e7488baa9bb73503c5de5b"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "96fa07cb673650fe884dc2fbdf5987cb"
server_key_iv: "cf9990fb8cb9d3e590ac1533d3eb3528"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.507787 29355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "370eb7b725e7488baa9bb73503c5de5b"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "96fa07cb673650fe884dc2fbdf5987cb"
server_key_iv: "cf9990fb8cb9d3e590ac1533d3eb3528"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.511783 29355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:49:40.514820 29373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.516170 29355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:40.516428 29355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "370eb7b725e7488baa9bb73503c5de5b"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "96fa07cb673650fe884dc2fbdf5987cb"
server_key_iv: "cf9990fb8cb9d3e590ac1533d3eb3528"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.516570 29355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:40.535604 29355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:40.539058 29355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:40.539331 29355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:40.549358 29355 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:42613
I20260811 18:49:40.549386 29442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:42613 every 8 connection(s)
I20260811 18:49:40.550534 29355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:40.553547 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:40.555004 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 29355
I20260811 18:49:40.555119 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:40.555449 28111 external_mini_cluster.cc:1428] Setting key bcd02de14d1c7ad4a267e8d1f573ade1
I20260811 18:49:40.560894 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Bootstrap starting.
I20260811 18:49:40.564049 29443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:40.564934 29443 log.cc:824] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:49:40 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:40.567757 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: No bootstrap required, opened a new log
I20260811 18:49:40.571400 29443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } }
I20260811 18:49:40.571799 29443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:40.571895 29443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 370eb7b725e7488baa9bb73503c5de5b, State: Initialized, Role: FOLLOWER
I20260811 18:49:40.572464 29443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } }
I20260811 18:49:40.572628 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.556992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.27.115.254:42613)
0811 18:49:40.557489 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.557508 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.557552 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:49:40.558422 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.558448 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.558905 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.559297 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.560025 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.561497 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.562617 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.563389 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.563568 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.566563 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.566570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.566587 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.566619 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.569294 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.569846 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.569852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.569857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.569978 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.570347 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.570351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.570352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.570824 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.571069 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.571344 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.571678 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":140,"threads_started":1}
I20260811 18:49:40.572642 29443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:40.572810 29443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:40.572940 29443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:40.574414 29443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } }
I20260811 18:49:40.574872 29443 leader_election.cc:302] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 370eb7b725e7488baa9bb73503c5de5b; no voters:
I20260811 18:49:40.575286 29443 leader_election.cc:288] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:40.575377 29449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:40.575577 29449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [term 1 LEADER]: Becoming Leader. State: Replica: 370eb7b725e7488baa9bb73503c5de5b, State: Running, Role: LEADER
I20260811 18:49:40.575929 29449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } }
I20260811 18:49:40.576804 29443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:40.578342 29450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "370eb7b725e7488baa9bb73503c5de5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } } }
I20260811 18:49:40.578491 29450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:40.578032 29451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 370eb7b725e7488baa9bb73503c5de5b. Latest consensus state: current_term: 1 leader_uuid: "370eb7b725e7488baa9bb73503c5de5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370eb7b725e7488baa9bb73503c5de5b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42613 } } }
I20260811 18:49:40.578784 29451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:40.579041 29462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:40.583158 29462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:40.591974 29462 catalog_manager.cc:1347] Generated new cluster ID: 97b80211ba9c4ef0a6c588ad75935b12
I20260811 18:49:40.592121 29462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:40.607708 29462 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:40.608807 29462 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:40.621528 29462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Generated new TSK 0
I20260811 18:49:40.622584 29462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:40.663321 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42613
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:40.819199 29475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:40.819540 29475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:40.819591 29475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:40.824046 29475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:40.824141 29475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:40.824185 29475 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:40.824221 29475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:40.829943 29475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:40.831162 29475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:40.832309 29475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:40.843869 29481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.843842 29480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.843835 29483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:40.844187 29475 server_base.cc:1034] running on GCE node
I20260811 18:49:40.844725 29475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:40.845475 29475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:40.846691 29475 hybrid_clock.cc:648] HybridClock initialized: now 1786474180846657 us; error 49 us; skew 500 ppm
Aug 11 18:49:40 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:40.849905 29475 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:40.850977 29475 webserver.cc:460] Webserver started at http://127.27.115.193:38735/ using document root <none> and password file <none>
I20260811 18:49:40.851603 29475 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:40.851701 29475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:40.851946 29475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:40.854223 29475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "cd734af9c62b4f3888c25774efd0fe97"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "bbcb3bd61ee66a2c45ed74c0de0213c6"
server_key_iv: "68d4c1805eaa19ea197e642a361e64d4"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.854856 29475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "cd734af9c62b4f3888c25774efd0fe97"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "bbcb3bd61ee66a2c45ed74c0de0213c6"
server_key_iv: "68d4c1805eaa19ea197e642a361e64d4"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.859138 29475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:40.861955 29490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.863531 29475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:49:40.863708 29475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "cd734af9c62b4f3888c25774efd0fe97"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-9jjr"
server_key: "bbcb3bd61ee66a2c45ed74c0de0213c6"
server_key_iv: "68d4c1805eaa19ea197e642a361e64d4"
server_key_version: "encryptionkey@0"
I20260811 18:49:40.863850 29475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:40.883667 29475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:40.887959 29475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:40.888203 29475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:40.889930 29475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:40.890022 29475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.890110 29475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:40.890200 29475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 18:49:40.903227 29475 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:43595
I20260811 18:49:40.903584 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:43595 every 8 connection(s)
Aug 11 18:49:40 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:40.904754 29475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:40.907223 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 29475
I20260811 18:49:40.907374 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:40.907680 28111 external_mini_cluster.cc:1428] Setting key 91e111fc34cc40066fc75eeaf42839ec
I20260811 18:49:40.910038 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.891141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56505 (local address 127.27.115.254:42613)
0811 18:49:40.891337 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.891344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.892511 (+ 1167us) server_negotiation.cc:407] Connection header received
0811 18:49:40.895317 (+ 2806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.895324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.895414 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.895545 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.897833 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.898845 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.901080 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.901514 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.901664 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.905599 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.905605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.905610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.905657 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.907541 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.908221 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.908226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.908228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.908332 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.908759 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.908762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.908764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.909005 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.909133 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.909713 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.909864 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:49:40.911121 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.891607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.193:56505)
0811 18:49:40.892314 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.892372 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.894945 (+ 2573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.895973 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.895987 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.896650 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:40.897583 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.897613 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.899054 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.899059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.900871 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.900892 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.901741 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.901745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.902018 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.902947 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:40.902972 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:40.905314 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:40.907698 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.907705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.907725 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.908093 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.908439 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.908445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.908450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.908646 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.909130 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:40.909138 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:40.909441 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.909668 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.910057 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":308,"threads_started":1}
I20260811 18:49:40.913015 29623 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42613
I20260811 18:49:40.913256 29623 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:40.914002 29623 heartbeater.cc:511] Master 127.27.115.254:42613 requested a full tablet report, sending...
I20260811 18:49:40.916210 29392 ts_manager.cc:194] Registered new tserver with Master: cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595)
I20260811 18:49:40.917475 29392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:56505
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:40.926951 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.915846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.193:33597)
0811 18:49:40.916222 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.916241 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.916350 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.918830 (+ 2480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.918835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.919135 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:40.919347 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.919357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.920415 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.920419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.921464 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.921482 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.922055 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.922057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.922217 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.922785 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:40.922798 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:40.923165 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:40.924998 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.925004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.925009 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.925358 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.925716 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.925719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.925720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.925812 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.926422 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:40.926424 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:40.926524 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.926629 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.926789 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260811 18:49:40.927134 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.918221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:33597 (local address 127.27.115.254:42613)
0811 18:49:40.918400 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.918406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.918425 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:49:40.918477 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.918484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.918545 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.918657 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.919508 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.920254 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.921667 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.921920 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.922018 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.923339 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.923343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.923345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.923371 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.924853 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.925489 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.925493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.925495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.925602 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.925917 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.925923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.925927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.926290 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.926408 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.926721 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.926875 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:40.946887 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42613
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:41.076741 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:41.077132 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:41.077250 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:41.082005 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:41.082119 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:41.082222 29635 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:41.082306 29635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:41.088606 29635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:41.089874 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:41.091068 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:41.102006 29640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:41.102380 29635 server_base.cc:1034] running on GCE node
W20260811 18:49:41.102006 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:41.102023 29641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:41.103103 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:41.103957 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:41.105206 29635 hybrid_clock.cc:648] HybridClock initialized: now 1786474181105177 us; error 44 us; skew 500 ppm
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:41.108773 29635 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:41.109965 29635 webserver.cc:460] Webserver started at http://127.27.115.194:39855/ using document root <none> and password file <none>
I20260811 18:49:41.110723 29635 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:41.110841 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:41.111124 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:41.113745 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "5eb8573c017c426d8ba844025087b40d"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "da08a5535e65bc86826a42be0623cd2c"
server_key_iv: "494a185160c35d46f4c38c5605f83863"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.114475 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "5eb8573c017c426d8ba844025087b40d"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "da08a5535e65bc86826a42be0623cd2c"
server_key_iv: "494a185160c35d46f4c38c5605f83863"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.119164 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:41.122553 29650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:41.124133 29635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:49:41.124383 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "5eb8573c017c426d8ba844025087b40d"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "da08a5535e65bc86826a42be0623cd2c"
server_key_iv: "494a185160c35d46f4c38c5605f83863"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.124552 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:41.138197 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:41.141741 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:41.141988 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:41.143899 29635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:41.144007 29635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:41.144091 29635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:41.144135 29635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:41.156459 29635 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:33109
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:41.156536 29789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:33109 every 8 connection(s)
I20260811 18:49:41.157608 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:41.161532 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.144857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:44101 (local address 127.27.115.254:42613)
0811 18:49:41.145014 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.145018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.146990 (+ 1972us) server_negotiation.cc:407] Connection header received
0811 18:49:41.148624 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.148630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.148730 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.148836 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.150723 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.151709 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.153254 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.153648 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.153797 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.157281 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.157285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.157287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.157318 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.159220 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.159791 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.159794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.159796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.159896 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.160380 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.160383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.160384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.160616 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.160699 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.161277 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.161400 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:49:41.162636 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.146064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.194:44101)
0811 18:49:41.146799 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.146863 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.148273 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.149076 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.149086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.149690 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:41.150477 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.150495 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.151901 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.151905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.153072 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.153084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.153869 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.153876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.154224 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.154988 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:41.155010 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:41.157061 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:41.159338 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.159342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.159354 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.159624 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.160064 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.160069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.160075 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.160260 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.160734 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:41.160740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:41.161027 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.161209 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.161661 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":135,"threads_started":1}
I20260811 18:49:41.164155 29790 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42613
I20260811 18:49:41.164353 29790 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:41.164882 29790 heartbeater.cc:511] Master 127.27.115.254:42613 requested a full tablet report, sending...
I20260811 18:49:41.166086 29392 ts_manager.cc:194] Registered new tserver with Master: 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109)
I20260811 18:49:41.167061 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 29635
I20260811 18:49:41.167193 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:41.167176 29392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:44101
I20260811 18:49:41.167521 28111 external_mini_cluster.cc:1428] Setting key f0228f79744f96aca84068942c09e706
I20260811 18:49:41.175616 29798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.166579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.194:51441)
0811 18:49:41.167011 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.167026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.167206 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.167606 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.167611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.167820 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:41.167975 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.167983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.168922 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.168943 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.169820 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.169835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.170529 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.170536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.170671 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.171249 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:41.171270 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:41.171728 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:41.173809 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.173812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.173814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.174034 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.174388 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.174390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.174391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.174481 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.175088 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:41.175091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:41.175204 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.175312 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.175460 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":142,"threads_started":1}
I20260811 18:49:41.175781 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.166643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:51441 (local address 127.27.115.254:42613)
0811 18:49:41.166801 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.166805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.167146 (+ 341us) server_negotiation.cc:407] Connection header received
0811 18:49:41.167339 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.167342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.167386 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.167436 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.168145 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.168736 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.170004 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.170371 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.170484 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.171876 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.171879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.171882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.171917 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.173673 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.174169 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.174175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.174180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.174286 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.174589 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.174595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.174600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.174954 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.175074 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.175411 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.175554 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:41.201624 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42613
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:41.323597 29801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:41.323957 29801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:41.324043 29801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:41.328620 29801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:41.328734 29801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:41.328816 29801 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:41.328902 29801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:41.334888 29801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:41.336163 29801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:41.337258 29801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:41.347926 29807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:41.347931 29808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:41.347980 29801 server_base.cc:1034] running on GCE node
W20260811 18:49:41.347931 29811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:41.348634 29801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:41.349399 29801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:41.350566 29801 hybrid_clock.cc:648] HybridClock initialized: now 1786474181350532 us; error 55 us; skew 500 ppm
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:41.353611 29801 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:41.354671 29801 webserver.cc:460] Webserver started at http://127.27.115.195:35775/ using document root <none> and password file <none>
I20260811 18:49:41.355276 29801 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:41.355346 29801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:41.355599 29801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:41.357791 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "3c5a75e6bcd947858d8a31afd39607b0"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "f9ac0f41f0fa086dd20cc32ad29a7b6c"
server_key_iv: "ccb8821c50773a80044de5f33e607092"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.358445 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "3c5a75e6bcd947858d8a31afd39607b0"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "f9ac0f41f0fa086dd20cc32ad29a7b6c"
server_key_iv: "ccb8821c50773a80044de5f33e607092"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.362983 29801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:41.366071 29818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:41.367547 29801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:49:41.367724 29801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "3c5a75e6bcd947858d8a31afd39607b0"
format_stamp: "Formatted at 2026-08-11 18:49:41 on dist-test-slave-9jjr"
server_key: "f9ac0f41f0fa086dd20cc32ad29a7b6c"
server_key_iv: "ccb8821c50773a80044de5f33e607092"
server_key_version: "encryptionkey@0"
I20260811 18:49:41.367883 29801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:41.379873 29801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:41.383754 29801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:41.384001 29801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:41.385610 29801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:41.385695 29801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:41.385757 29801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:41.385811 29801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:41.398221 29801 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:39265
I20260811 18:49:41.398303 29950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:39265 every 8 connection(s)
I20260811 18:49:41.399473 29801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:41.399856 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 29801
I20260811 18:49:41.400019 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:41.400350 28111 external_mini_cluster.cc:1428] Setting key d386256bdad02247f826e900f8b05146
I20260811 18:49:41.401531 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.386652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:56815 (local address 127.27.115.254:42613)
0811 18:49:41.386788 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.386792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.387987 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 18:49:41.389167 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.389170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.389234 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.389325 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.390758 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.391476 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.392830 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.393176 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.393285 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.396939 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.396944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.396946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.396981 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.398830 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.399429 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.399436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.399441 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.399555 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.400045 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.400048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.400050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.400344 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.400455 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.401192 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.401354 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:49:41.402657 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.387264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.195:56815)
0811 18:49:41.387817 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.387863 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.388873 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.389619 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.389626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.389963 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:41.390604 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.390621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.391613 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.391617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.392694 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.392705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.393428 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.393436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.393714 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.394582 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:41.394612 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:41.396727 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:41.399001 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.399006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.399019 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.399301 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.399687 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.399691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.399693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.399907 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.400550 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:41.400557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:41.400894 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.401144 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.401501 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":153,"threads_started":1}
I20260811 18:49:41.403932 29952 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42613
I20260811 18:49:41.404114 29952 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:41.404666 29952 heartbeater.cc:511] Master 127.27.115.254:42613 requested a full tablet report, sending...
I20260811 18:49:41.405812 29393 ts_manager.cc:194] Registered new tserver with Master: 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265)
I20260811 18:49:41.406620 29393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:56815
I20260811 18:49:41.414649 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.406518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:59589 (local address 127.27.115.254:42613)
0811 18:49:41.406663 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.406668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.406930 (+ 262us) server_negotiation.cc:407] Connection header received
0811 18:49:41.407040 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.407043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.407096 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.407159 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.407673 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.408414 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.409393 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.409642 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.409729 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.410979 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.410983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.410985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.411022 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.412412 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.413003 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.413007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.413009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.413133 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.413588 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.413592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.413594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.413844 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.413976 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.414341 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.414481 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:49:41.414670 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.406439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.27.115.195:59589)
0811 18:49:41.406822 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.406838 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.406954 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.407204 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.407208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.407441 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:41.407548 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.407555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.408652 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.408657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.409269 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.409279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.409762 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.409765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.409838 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.410446 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:41.410458 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:41.410817 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:41.412567 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.412574 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.412579 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.412826 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.413298 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.413304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.413305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.413426 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.413992 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:41.413995 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:41.414106 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.414261 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.414440 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":128,"threads_started":1}
I20260811 18:49:41.415319 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:41.445523 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.436515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.27.115.254:42613)
0811 18:49:41.436678 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.436682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.436697 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:49:41.436745 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.436748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.436805 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.436889 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.437742 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.438421 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.439331 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.439547 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.439634 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.442330 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.442335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.442337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.442382 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.443828 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.444284 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.444287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.444288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.444373 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.444636 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.444639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.444640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.444851 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.444962 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.445184 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.445296 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:49:41.448886 29393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:41.452385 29393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:41.468075 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.461401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33109 (local address 127.0.0.1:43286)
0811 18:49:41.462220 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.462281 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.462439 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.463423 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.463430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.463458 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.463743 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.463754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.464895 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.464900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.466361 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.466371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.467464 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.467468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.467582 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.467634 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.467729 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.467870 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":25,"spinlock_wait_cycles":4864,"thread_start_us":124,"threads_started":1}
I20260811 18:49:41.468425 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.461606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43595 (local address 127.0.0.1:45018)
0811 18:49:41.462692 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.462705 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.462784 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.463465 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.463469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.463488 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.463760 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.463774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.465125 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.465129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.467021 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.467030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.467964 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.467968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.468107 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.468131 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.468205 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.468290 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":87,"thread_start_us":123,"threads_started":1}
I20260811 18:49:41.468756 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.462353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.27.115.193:43595)
0811 18:49:41.462969 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.462976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.462998 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:49:41.463098 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.463103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.463313 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.463709 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.463864 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.465010 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.467154 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.467770 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.467980 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.468138 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.468474 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.468606 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":98,"threads_started":1}
I20260811 18:49:41.468874 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.461605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.27.115.194:33109)
0811 18:49:41.462743 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.462753 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.462808 (+ 55us) server_negotiation.cc:407] Connection header received
0811 18:49:41.462893 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.462900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.463218 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.463477 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.463896 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.464778 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.466487 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.467313 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.468426 (+ 1113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.468573 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.468623 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.468715 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":125,"threads_started":1}
I20260811 18:49:41.469789 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.462350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.27.115.195:39265)
0811 18:49:41.463022 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.463034 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.463413 (+ 379us) server_negotiation.cc:407] Connection header received
0811 18:49:41.463547 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.463552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.463746 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.464211 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.464347 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.465465 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.466830 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.467640 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.467800 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.467940 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.469522 (+ 1582us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.469643 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":100,"threads_started":1}
I20260811 18:49:41.469935 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.461822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39265 (local address 127.0.0.1:40750)
0811 18:49:41.463209 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.463222 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.463312 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.463935 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.463938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.463958 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.464044 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.464050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.465583 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.465589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.466684 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.466696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.468060 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.468064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.468773 (+ 709us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.468828 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.468912 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.468989 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":697,"thread_start_us":91,"threads_started":1}
I20260811 18:49:41.471832 29546 tablet_service.cc:1467] Processing CreateTablet for tablet e3f5e609bcd343169af1faa21f41e5dc (DEFAULT_TABLE table=test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.471872 29706 tablet_service.cc:1467] Processing CreateTablet for tablet e3f5e609bcd343169af1faa21f41e5dc (DEFAULT_TABLE table=test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.472601 29875 tablet_service.cc:1467] Processing CreateTablet for tablet e3f5e609bcd343169af1faa21f41e5dc (DEFAULT_TABLE table=test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.473043 29706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f5e609bcd343169af1faa21f41e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.473044 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f5e609bcd343169af1faa21f41e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.473589 29875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f5e609bcd343169af1faa21f41e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.481551 29980 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Bootstrap starting.
I20260811 18:49:41.481551 29979 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Bootstrap starting.
I20260811 18:49:41.482435 29978 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Bootstrap starting.
I20260811 18:49:41.484117 29979 tablet_bootstrap.cc:654] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.484117 29980 tablet_bootstrap.cc:654] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.484223 29978 tablet_bootstrap.cc:654] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.485014 29978 log.cc:824] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:41.485181 29980 log.cc:824] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:41.485178 29979 log.cc:824] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:41.487241 29979 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: No bootstrap required, opened a new log
I20260811 18:49:41.487268 29978 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: No bootstrap required, opened a new log
I20260811 18:49:41.487483 29979 ts_tablet_manager.cc:1397] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:49:41.487491 29978 ts_tablet_manager.cc:1397] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:49:41.487591 29980 tablet_bootstrap.cc:492] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: No bootstrap required, opened a new log
I20260811 18:49:41.487795 29980 ts_tablet_manager.cc:1397] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 18:49:41.490819 29979 raft_consensus.cc:348] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.491091 29979 raft_consensus.cc:374] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.491158 29979 raft_consensus.cc:729] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5a75e6bcd947858d8a31afd39607b0, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.491621 29979 consensus_queue.cc:260] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.491873 29978 raft_consensus.cc:348] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.492142 29978 raft_consensus.cc:374] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.492200 29978 raft_consensus.cc:729] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd734af9c62b4f3888c25774efd0fe97, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.491873 29980 raft_consensus.cc:348] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.492543 29952 heartbeater.cc:503] Master 127.27.115.254:42613 was elected leader, sending a full tablet report...
I20260811 18:49:41.492588 29980 raft_consensus.cc:374] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.492648 29980 raft_consensus.cc:729] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb8573c017c426d8ba844025087b40d, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.492797 29979 ts_tablet_manager.cc:1428] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 18:49:41.493073 29980 consensus_queue.cc:260] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.493813 29790 heartbeater.cc:503] Master 127.27.115.254:42613 was elected leader, sending a full tablet report...
I20260811 18:49:41.493914 29978 consensus_queue.cc:260] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.494095 29980 ts_tablet_manager.cc:1428] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260811 18:49:41.494837 29623 heartbeater.cc:503] Master 127.27.115.254:42613 was elected leader, sending a full tablet report...
I20260811 18:49:41.494870 29978 ts_tablet_manager.cc:1428] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:49:41.496381 29986 raft_consensus.cc:482] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:41.496523 29986 raft_consensus.cc:504] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.497964 29986 leader_election.cc:288] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595), 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265)
I20260811 18:49:41.503587 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.498782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:55609 (local address 127.27.115.193:43595)
0811 18:49:41.498905 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.498909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.498923 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:49:41.498969 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.498971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.499030 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.499109 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.499500 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.500177 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.501866 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.502604 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.502738 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.502797 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.503346 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.503429 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:49:41.503587 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.498212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43595 (local address 127.27.115.194:55609)
0811 18:49:41.498514 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.498530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.498616 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.499154 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.499157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.499180 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.499358 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.499368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.500332 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.500335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.501734 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.501744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.502734 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.502737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.503179 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.503208 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.503327 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.503430 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":316}
I20260811 18:49:41.504510 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f5e609bcd343169af1faa21f41e5dc" candidate_uuid: "5eb8573c017c426d8ba844025087b40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734af9c62b4f3888c25774efd0fe97" is_pre_election: true
I20260811 18:49:41.504967 29566 raft_consensus.cc:2454] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb8573c017c426d8ba844025087b40d in term 0.
I20260811 18:49:41.505373 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.498330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:44245 (local address 127.27.115.195:39265)
0811 18:49:41.498465 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.498470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.499100 (+ 630us) server_negotiation.cc:407] Connection header received
0811 18:49:41.499161 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.499164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.499207 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.499297 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.499902 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.500642 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.503897 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.504491 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.504598 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.504654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.505107 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.505207 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:49:41.505475 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.498222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39265 (local address 127.27.115.194:44245)
0811 18:49:41.498974 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.498987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.499090 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.499328 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.499330 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.499346 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.499671 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.499680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.500777 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.500780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.503583 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.503594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.504766 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.504769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.504947 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.504975 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.505058 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.505137 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":92,"thread_start_us":142,"threads_started":1}
I20260811 18:49:41.505630 29651 leader_election.cc:302] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb8573c017c426d8ba844025087b40d, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.505972 29986 raft_consensus.cc:2781] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:41.506072 29986 raft_consensus.cc:482] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:41.506170 29986 raft_consensus.cc:3037] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.506182 29898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f5e609bcd343169af1faa21f41e5dc" candidate_uuid: "5eb8573c017c426d8ba844025087b40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5a75e6bcd947858d8a31afd39607b0" is_pre_election: true
I20260811 18:49:41.506453 29898 raft_consensus.cc:2454] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb8573c017c426d8ba844025087b40d in term 0.
I20260811 18:49:41.507529 29986 raft_consensus.cc:504] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.507948 29986 leader_election.cc:288] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [CANDIDATE]: Term 1 election: Requested vote from peers cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595), 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265)
I20260811 18:49:41.508315 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f5e609bcd343169af1faa21f41e5dc" candidate_uuid: "5eb8573c017c426d8ba844025087b40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734af9c62b4f3888c25774efd0fe97"
I20260811 18:49:41.508347 29898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f5e609bcd343169af1faa21f41e5dc" candidate_uuid: "5eb8573c017c426d8ba844025087b40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5a75e6bcd947858d8a31afd39607b0"
I20260811 18:49:41.508461 29566 raft_consensus.cc:3037] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.508460 29898 raft_consensus.cc:3037] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.509868 29566 raft_consensus.cc:2454] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb8573c017c426d8ba844025087b40d in term 1.
I20260811 18:49:41.509868 29898 raft_consensus.cc:2454] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb8573c017c426d8ba844025087b40d in term 1.
I20260811 18:49:41.510252 29651 leader_election.cc:302] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb8573c017c426d8ba844025087b40d, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.510589 29986 raft_consensus.cc:2781] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:41.510862 29986 raft_consensus.cc:686] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 1 LEADER]: Becoming Leader. State: Replica: 5eb8573c017c426d8ba844025087b40d, State: Running, Role: LEADER
I20260811 18:49:41.511282 29986 consensus_queue.cc:237] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.515169 29392 catalog_manager.cc:5495] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5eb8573c017c426d8ba844025087b40d (127.27.115.194). New cstate: current_term: 1 leader_uuid: "5eb8573c017c426d8ba844025087b40d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260811 18:49:41.544922 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.540740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.27.115.194:33109)
0811 18:49:41.540909 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.540913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.540930 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:49:41.541154 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.541157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.541218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.541324 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:41.541623 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.542173 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.543105 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.543361 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.543449 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.543887 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.544402 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.544498 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.544622 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.544682 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:49:41.551913 29566 raft_consensus.cc:1261] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Refusing update from remote peer 5eb8573c017c426d8ba844025087b40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.552778 29986 consensus_queue.cc:1035] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.554116 29898 raft_consensus.cc:1261] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Refusing update from remote peer 5eb8573c017c426d8ba844025087b40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.554890 29989 consensus_queue.cc:1035] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.562845 29994 mvcc.cc:205] Tried to move back new op lower bound from 7317398247626899456 to 7317398247473467392. Current Snapshot: MvccSnapshot[applied={T|T < 7317398247626899456}]
I20260811 18:49:41.563674 29995 mvcc.cc:205] Tried to move back new op lower bound from 7317398247626899456 to 7317398247473467392. Current Snapshot: MvccSnapshot[applied={T|T < 7317398247626899456}]
I20260811 18:49:41.567638 29997 mvcc.cc:205] Tried to move back new op lower bound from 7317398247626899456 to 7317398247473467392. Current Snapshot: MvccSnapshot[applied={T|T < 7317398247626899456 or (T in {7317398247626899456})}]
I20260811 18:49:41.579238 29393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:41.579612 29393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:41.583287 29393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Sending DeleteTablet for 3 replicas of tablet e3f5e609bcd343169af1faa21f41e5dc
I20260811 18:49:41.584528 29706 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f5e609bcd343169af1faa21f41e5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:43286
I20260811 18:49:41.585372 30004 tablet_replica.cc:330] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: stopping tablet replica
I20260811 18:49:41.585294 29546 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f5e609bcd343169af1faa21f41e5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:45018
I20260811 18:49:41.585776 30004 raft_consensus.cc:2229] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:49:41.586325 30004 raft_consensus.cc:2258] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:41.586930 30005 tablet_replica.cc:330] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: stopping tablet replica
I20260811 18:49:41.587148 29393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:49:41.587260 30005 raft_consensus.cc:2229] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:41.587595 30005 raft_consensus.cc:2258] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:49:41.587666 29393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:41.587613 29875 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f5e609bcd343169af1faa21f41e5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:40750
I20260811 18:49:41.588827 30006 tablet_replica.cc:330] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: stopping tablet replica
I20260811 18:49:41.589249 30006 raft_consensus.cc:2229] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:41.589636 30006 raft_consensus.cc:2258] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:41.591415 30006 ts_tablet_manager.cc:1905] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:41.593101 30004 ts_tablet_manager.cc:1905] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:41.593426 30005 ts_tablet_manager.cc:1905] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:41.593916 29706 tablet_service.cc:1467] Processing CreateTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (DEFAULT_TABLE table=test-table [id=d89dc2b21ff24046af4bf22abcac75c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.594291 29706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.594792 30006 ts_tablet_manager.cc:1918] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:49:41.594902 30006 log.cc:1196] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/e3f5e609bcd343169af1faa21f41e5dc
I20260811 18:49:41.595299 30006 ts_tablet_manager.cc:1939] T e3f5e609bcd343169af1faa21f41e5dc P 3c5a75e6bcd947858d8a31afd39607b0: Deleting consensus metadata
I20260811 18:49:41.597018 29378 catalog_manager.cc:4841] TS 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265): tablet e3f5e609bcd343169af1faa21f41e5dc (table test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]) successfully deleted
I20260811 18:49:41.597112 29875 tablet_service.cc:1467] Processing CreateTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (DEFAULT_TABLE table=test-table [id=d89dc2b21ff24046af4bf22abcac75c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.597384 29875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.597452 29980 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Bootstrap starting.
I20260811 18:49:41.599184 29980 tablet_bootstrap.cc:654] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.600113 29979 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Bootstrap starting.
I20260811 18:49:41.601184 29546 tablet_service.cc:1467] Processing CreateTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (DEFAULT_TABLE table=test-table [id=d89dc2b21ff24046af4bf22abcac75c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:41.601464 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.601516 29979 tablet_bootstrap.cc:654] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.601668 29980 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: No bootstrap required, opened a new log
I20260811 18:49:41.601820 29980 ts_tablet_manager.cc:1397] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:41.602398 30004 ts_tablet_manager.cc:1918] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:49:41.602766 30005 ts_tablet_manager.cc:1918] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:49:41.603009 30005 log.cc:1196] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/e3f5e609bcd343169af1faa21f41e5dc
I20260811 18:49:41.603439 30005 ts_tablet_manager.cc:1939] T e3f5e609bcd343169af1faa21f41e5dc P cd734af9c62b4f3888c25774efd0fe97: Deleting consensus metadata
I20260811 18:49:41.602650 29980 raft_consensus.cc:348] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.603579 29980 raft_consensus.cc:374] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.603612 29979 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: No bootstrap required, opened a new log
I20260811 18:49:41.603628 29980 raft_consensus.cc:729] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb8573c017c426d8ba844025087b40d, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.603688 29979 ts_tablet_manager.cc:1397] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 18:49:41.603775 29980 consensus_queue.cc:260] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.604076 29980 ts_tablet_manager.cc:1428] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:49:41.604501 29979 raft_consensus.cc:348] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.604619 29979 raft_consensus.cc:374] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.604648 29979 raft_consensus.cc:729] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5a75e6bcd947858d8a31afd39607b0, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.604777 29979 consensus_queue.cc:260] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.605118 29979 ts_tablet_manager.cc:1428] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:41.605259 30004 log.cc:1196] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/e3f5e609bcd343169af1faa21f41e5dc
I20260811 18:49:41.605710 30004 ts_tablet_manager.cc:1939] T e3f5e609bcd343169af1faa21f41e5dc P 5eb8573c017c426d8ba844025087b40d: Deleting consensus metadata
I20260811 18:49:41.606184 29978 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Bootstrap starting.
I20260811 18:49:41.606998 29374 catalog_manager.cc:4841] TS cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595): tablet e3f5e609bcd343169af1faa21f41e5dc (table test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]) successfully deleted
I20260811 18:49:41.607956 29377 catalog_manager.cc:4841] TS 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109): tablet e3f5e609bcd343169af1faa21f41e5dc (table test-table [id=668e4b9fc9554a5fa82ad1a731b32d74]) successfully deleted
I20260811 18:49:41.608054 29978 tablet_bootstrap.cc:654] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.609758 29978 tablet_bootstrap.cc:492] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: No bootstrap required, opened a new log
I20260811 18:49:41.609871 29978 ts_tablet_manager.cc:1397] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:41.610873 29978 raft_consensus.cc:348] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.611006 29978 raft_consensus.cc:374] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.611049 29978 raft_consensus.cc:729] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd734af9c62b4f3888c25774efd0fe97, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.611203 29978 consensus_queue.cc:260] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.611558 29978 ts_tablet_manager.cc:1428] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260811 18:49:41.651203 29954 tablet.cc:2369] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:41.656766 29625 tablet.cc:2369] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:41.658989 29791 tablet.cc:2369] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:49:41.702503 29985 raft_consensus.cc:482] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:41.702812 29985 raft_consensus.cc:504] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.704326 29985 leader_election.cc:288] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595), 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109)
I20260811 18:49:41.710562 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.705505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:46527 (local address 127.27.115.193:43595)
0811 18:49:41.705673 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.705679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.705706 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:49:41.705762 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.705765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.705811 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.705950 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.706579 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.707282 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.708840 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.709633 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.709810 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.709907 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.710281 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.710385 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:49:41.711118 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.704727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43595 (local address 127.27.115.195:46527)
0811 18:49:41.705198 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.705213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.705290 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.705980 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.705983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.706029 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.706408 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.706421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.707410 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.707415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.708667 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.708682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.709891 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.709896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.710074 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.710103 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.710746 (+ 643us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.710877 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":119,"threads_started":1}
I20260811 18:49:41.711741 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d0a328cf2b4f5c9de63265b7e7ddc3" candidate_uuid: "3c5a75e6bcd947858d8a31afd39607b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734af9c62b4f3888c25774efd0fe97" is_pre_election: true
I20260811 18:49:41.711930 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.704941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:59269 (local address 127.27.115.194:33109)
0811 18:49:41.705289 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.705295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.705320 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:49:41.705414 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.705417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.705479 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.705603 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.706217 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.706960 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.709156 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.710047 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.710282 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.710372 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.711509 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.711663 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260811 18:49:41.711952 29566 raft_consensus.cc:2454] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c5a75e6bcd947858d8a31afd39607b0 in term 0.
I20260811 18:49:41.711195 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.704674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33109 (local address 127.27.115.195:59269)
0811 18:49:41.705134 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.705153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.705288 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.705666 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.705702 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.705729 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.706010 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.706022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.707125 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.707129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.708878 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.708920 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.710472 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.710476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.710695 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.710726 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.710875 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.711000 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359}
I20260811 18:49:41.712643 29819 leader_election.cc:302] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5a75e6bcd947858d8a31afd39607b0, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.713106 29985 raft_consensus.cc:2781] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:41.713197 29985 raft_consensus.cc:482] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:41.713244 29985 raft_consensus.cc:3037] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.713744 29720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d0a328cf2b4f5c9de63265b7e7ddc3" candidate_uuid: "3c5a75e6bcd947858d8a31afd39607b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb8573c017c426d8ba844025087b40d" is_pre_election: true
I20260811 18:49:41.714072 29720 raft_consensus.cc:2454] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c5a75e6bcd947858d8a31afd39607b0 in term 0.
I20260811 18:49:41.715420 29985 raft_consensus.cc:504] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.715891 29985 leader_election.cc:288] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [CANDIDATE]: Term 1 election: Requested vote from peers cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595), 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109)
I20260811 18:49:41.716414 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d0a328cf2b4f5c9de63265b7e7ddc3" candidate_uuid: "3c5a75e6bcd947858d8a31afd39607b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd734af9c62b4f3888c25774efd0fe97"
I20260811 18:49:41.716583 29566 raft_consensus.cc:3037] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.716571 29720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d0a328cf2b4f5c9de63265b7e7ddc3" candidate_uuid: "3c5a75e6bcd947858d8a31afd39607b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb8573c017c426d8ba844025087b40d"
I20260811 18:49:41.716842 29720 raft_consensus.cc:3037] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.718031 29566 raft_consensus.cc:2454] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c5a75e6bcd947858d8a31afd39607b0 in term 1.
I20260811 18:49:41.718564 29819 leader_election.cc:302] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5a75e6bcd947858d8a31afd39607b0, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.718814 29985 raft_consensus.cc:2781] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:41.719130 29985 raft_consensus.cc:686] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 LEADER]: Becoming Leader. State: Replica: 3c5a75e6bcd947858d8a31afd39607b0, State: Running, Role: LEADER
I20260811 18:49:41.719451 29985 consensus_queue.cc:237] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.720372 29720 raft_consensus.cc:2454] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c5a75e6bcd947858d8a31afd39607b0 in term 1.
I20260811 18:49:41.722381 29392 catalog_manager.cc:5495] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "3c5a75e6bcd947858d8a31afd39607b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } health_report { overall_health: HEALTHY } } }
Aug 11 18:49:41 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:41.776715 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.764503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42613 (local address 127.0.0.1:53040)
0811 18:49:41.765036 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.765052 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.765223 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.765566 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.765569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.766292 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:41.766438 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.766448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.767327 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.767334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.768063 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.768077 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.768624 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.768627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.768717 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.769568 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:41.769581 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:41.771446 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:41.774297 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.774314 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.774321 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.774676 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.775148 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:41.775151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:41.775152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:41.775262 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:41.776035 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:41.776042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:41.776185 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.776293 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.776444 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":229,"threads_started":1}
I20260811 18:49:41.776759 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.764598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.27.115.254:42613)
0811 18:49:41.764814 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.764820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.765156 (+ 336us) server_negotiation.cc:407] Connection header received
0811 18:49:41.765383 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.765385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.765430 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.765505 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.766613 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.767147 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.768237 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.768474 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.768612 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.771787 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:41.771803 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:41.771807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:41.771853 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:41.773988 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.774813 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.774818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.774820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.774950 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:41.775424 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:41.775428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:41.775429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:41.775839 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:41.776030 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.776419 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.776608 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:49:41.783288 29392 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:53040:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:49:41.784346 29392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:41.791105 29706 tablet_service.cc:1467] Processing CreateTablet for tablet f36f129dedda4a51906f3ab551d8f9da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f075a5e8895f405bad0e3984dbd66a54]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:41.791471 29546 tablet_service.cc:1467] Processing CreateTablet for tablet f36f129dedda4a51906f3ab551d8f9da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f075a5e8895f405bad0e3984dbd66a54]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:41.791725 29706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f129dedda4a51906f3ab551d8f9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.791908 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f129dedda4a51906f3ab551d8f9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.792035 29875 tablet_service.cc:1467] Processing CreateTablet for tablet f36f129dedda4a51906f3ab551d8f9da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f075a5e8895f405bad0e3984dbd66a54]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:41.792474 29875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f129dedda4a51906f3ab551d8f9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:41.795825 29980 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: Bootstrap starting.
I20260811 18:49:41.795908 29978 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: Bootstrap starting.
I20260811 18:49:41.796140 29979 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: Bootstrap starting.
I20260811 18:49:41.796995 29980 tablet_bootstrap.cc:654] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.797389 29979 tablet_bootstrap.cc:654] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.797441 29978 tablet_bootstrap.cc:654] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:41.799060 29980 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: No bootstrap required, opened a new log
I20260811 18:49:41.799180 29980 ts_tablet_manager.cc:1397] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:49:41.799371 29979 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: No bootstrap required, opened a new log
I20260811 18:49:41.799485 29979 ts_tablet_manager.cc:1397] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:49:41.799732 29980 raft_consensus.cc:348] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.799890 29980 raft_consensus.cc:374] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.799929 29980 raft_consensus.cc:729] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb8573c017c426d8ba844025087b40d, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.800086 29980 consensus_queue.cc:260] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.800092 29979 raft_consensus.cc:348] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.800254 29979 raft_consensus.cc:374] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.800300 29979 raft_consensus.cc:729] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5a75e6bcd947858d8a31afd39607b0, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.800427 29980 ts_tablet_manager.cc:1428] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:41.800750 29978 tablet_bootstrap.cc:492] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: No bootstrap required, opened a new log
I20260811 18:49:41.800705 29979 consensus_queue.cc:260] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.800767 29986 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.800863 29978 ts_tablet_manager.cc:1397] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:49:41.800909 29986 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.801087 29979 ts_tablet_manager.cc:1428] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:49:41.801362 29985 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.801518 29985 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.801458 29978 raft_consensus.cc:348] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.801571 29978 raft_consensus.cc:374] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:41.801604 29978 raft_consensus.cc:729] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd734af9c62b4f3888c25774efd0fe97, State: Initialized, Role: FOLLOWER
I20260811 18:49:41.801745 29978 consensus_queue.cc:260] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.802047 29978 ts_tablet_manager.cc:1428] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:49:41.802162 29987 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.802423 29987 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.826814 29987 raft_consensus.cc:482] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:41.827015 29987 raft_consensus.cc:504] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.828336 29987 leader_election.cc:288] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109), 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265)
I20260811 18:49:41.834668 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.828982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:40267 (local address 127.27.115.195:39265)
0811 18:49:41.829200 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.829205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.829502 (+ 297us) server_negotiation.cc:407] Connection header received
0811 18:49:41.829659 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.829663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.829711 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.829798 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.830478 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.831285 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.833288 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.834001 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.834161 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.834226 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.834474 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.834543 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 18:49:41.834728 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.828830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39265 (local address 127.27.115.193:40267)
0811 18:49:41.829405 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.829421 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.829511 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.829935 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.829938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.829984 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.830297 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.830306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.831432 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.831436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.833081 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.833094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.834180 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.834184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.834316 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.834348 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.834438 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.834526 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":152,"spinlock_wait_cycles":7424,"thread_start_us":234,"threads_started":1}
I20260811 18:49:41.835489 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.828566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33109 (local address 127.27.115.193:42217)
0811 18:49:41.829066 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.829200 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.829498 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.829935 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.829938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.829982 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:41.830297 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.830306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.831432 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.831436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.833078 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.833094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.834946 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.834951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.835127 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.835174 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.835269 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.835345 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":170,"spinlock_wait_cycles":31872,"thread_start_us":149,"threads_started":1}
I20260811 18:49:41.835539 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.828708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:42217 (local address 127.27.115.194:33109)
0811 18:49:41.828866 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.828873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.829387 (+ 514us) server_negotiation.cc:407] Connection header received
0811 18:49:41.829637 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.829640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.829703 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.829798 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:41.830478 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.831285 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.833288 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.834001 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.834171 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.834226 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.835325 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.835421 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:49:41.835671 29898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f129dedda4a51906f3ab551d8f9da" candidate_uuid: "cd734af9c62b4f3888c25774efd0fe97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5a75e6bcd947858d8a31afd39607b0" is_pre_election: true
I20260811 18:49:41.835844 29898 raft_consensus.cc:2454] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd734af9c62b4f3888c25774efd0fe97 in term 0.
I20260811 18:49:41.836208 29720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f129dedda4a51906f3ab551d8f9da" candidate_uuid: "cd734af9c62b4f3888c25774efd0fe97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb8573c017c426d8ba844025087b40d" is_pre_election: true
I20260811 18:49:41.836369 29720 raft_consensus.cc:2454] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd734af9c62b4f3888c25774efd0fe97 in term 0.
I20260811 18:49:41.836359 29494 leader_election.cc:302] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5a75e6bcd947858d8a31afd39607b0, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.836715 29987 raft_consensus.cc:2781] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:41.836808 29987 raft_consensus.cc:482] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:41.836853 29987 raft_consensus.cc:3037] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.838305 29987 raft_consensus.cc:504] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.838744 29987 leader_election.cc:288] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [CANDIDATE]: Term 1 election: Requested vote from peers 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109), 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265)
I20260811 18:49:41.839191 29720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f129dedda4a51906f3ab551d8f9da" candidate_uuid: "cd734af9c62b4f3888c25774efd0fe97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eb8573c017c426d8ba844025087b40d"
I20260811 18:49:41.839294 29898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f129dedda4a51906f3ab551d8f9da" candidate_uuid: "cd734af9c62b4f3888c25774efd0fe97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5a75e6bcd947858d8a31afd39607b0"
I20260811 18:49:41.839362 29720 raft_consensus.cc:3037] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.839480 29898 raft_consensus.cc:3037] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:41.840663 29720 raft_consensus.cc:2454] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd734af9c62b4f3888c25774efd0fe97 in term 1.
I20260811 18:49:41.841121 29493 leader_election.cc:302] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb8573c017c426d8ba844025087b40d, cd734af9c62b4f3888c25774efd0fe97; no voters:
I20260811 18:49:41.841228 29898 raft_consensus.cc:2454] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd734af9c62b4f3888c25774efd0fe97 in term 1.
I20260811 18:49:41.841665 29987 raft_consensus.cc:2781] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:41.841985 29987 raft_consensus.cc:686] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [term 1 LEADER]: Becoming Leader. State: Replica: cd734af9c62b4f3888c25774efd0fe97, State: Running, Role: LEADER
I20260811 18:49:41.842348 29987 consensus_queue.cc:237] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } }
I20260811 18:49:41.844082 30028 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: TxnStatusTablet state changed. Reason: New leader cd734af9c62b4f3888c25774efd0fe97. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.844240 30028 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:41.844663 30032 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:49:41.844915 30032 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:49:41.845645 29392 catalog_manager.cc:5495] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193). New cstate: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260811 18:49:41.867031 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.860517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43595 (local address 127.0.0.1:45026)
0811 18:49:41.860869 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.860885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.861151 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.861654 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.861658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.861669 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:41.861803 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.861810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.863053 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.863058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.864179 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.864193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.865173 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.865178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.865286 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.865369 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.866542 (+ 1173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.866573 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.866684 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.866809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260811 18:49:41.867089 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.860628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.27.115.193:43595)
0811 18:49:41.860768 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.860772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.860989 (+ 217us) server_negotiation.cc:407] Connection header received
0811 18:49:41.861380 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.861383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.861476 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.861593 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:41.861985 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.862744 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.864364 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.864775 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.865189 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.865624 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.866414 (+ 790us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.866528 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.866776 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.866903 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:49:41.871505 29898 raft_consensus.cc:1261] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Refusing update from remote peer cd734af9c62b4f3888c25774efd0fe97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.871680 29720 raft_consensus.cc:1261] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Refusing update from remote peer cd734af9c62b4f3888c25774efd0fe97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.872354 30028 consensus_queue.cc:1035] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.872692 29987 consensus_queue.cc:1035] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.874641 29985 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: TxnStatusTablet state changed. Reason: New leader cd734af9c62b4f3888c25774efd0fe97. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.874805 29985 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.874863 29986 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: TxnStatusTablet state changed. Reason: New leader cd734af9c62b4f3888c25774efd0fe97. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.875016 29986 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.878720 29985 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.878892 29985 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 3c5a75e6bcd947858d8a31afd39607b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.879155 29986 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.879292 29986 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P 5eb8573c017c426d8ba844025087b40d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:41.879918 29987 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.880085 29987 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:41.882936 30028 tablet_replica.cc:439] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd734af9c62b4f3888c25774efd0fe97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 } } peers { permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 } } peers { permanent_uuid: "3c5a75e6bcd947858d8a31afd39607b0" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39265 } } }
I20260811 18:49:41.883103 30028 tablet_replica.cc:442] T f36f129dedda4a51906f3ab551d8f9da P cd734af9c62b4f3888c25774efd0fe97: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:41.891008 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.886012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.27.115.195:39265)
0811 18:49:41.886190 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.886195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.886303 (+ 108us) server_negotiation.cc:407] Connection header received
0811 18:49:41.886595 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.886598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.886677 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.886813 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:41.887050 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.887903 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.888809 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.889271 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.889386 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.889876 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.890508 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.890636 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.890788 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.890872 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:49:41.903903 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.900222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43595 (local address 127.27.115.195:53323)
0811 18:49:41.900491 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.900508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.900642 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.900930 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.900933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.900940 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:41.901083 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.901088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.901844 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.901848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.902522 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.902532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.903012 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.903016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.903125 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.903198 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.903586 (+ 388us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.903595 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.903688 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.903780 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 18:49:41.903903 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.900372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:53323 (local address 127.27.115.193:43595)
0811 18:49:41.900522 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.900529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.900549 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:49:41.900768 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.900771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.900820 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.900909 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:41.901198 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.901725 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.902658 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.902886 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.903004 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.903389 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.903509 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.903581 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.903703 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.903762 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:49:41.914390 29720 raft_consensus.cc:1261] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Refusing update from remote peer 3c5a75e6bcd947858d8a31afd39607b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.914471 29566 raft_consensus.cc:1261] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Refusing update from remote peer 3c5a75e6bcd947858d8a31afd39607b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:41.915066 29985 consensus_queue.cc:1035] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb8573c017c426d8ba844025087b40d" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.915395 30012 consensus_queue.cc:1035] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd734af9c62b4f3888c25774efd0fe97" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:41.948235 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.943416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56635 (local address 127.27.115.195:39265)
0811 18:49:41.943534 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.943538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.944008 (+ 470us) server_negotiation.cc:407] Connection header received
0811 18:49:41.944116 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.944119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.944178 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.944266 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:41.944700 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.945399 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.946862 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.947272 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.947393 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.947744 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.947849 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.947926 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:41.948032 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 18:49:41.948094 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:49:41.948299 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.943282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39265 (local address 127.27.115.193:56635)
0811 18:49:41.943775 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:41.943789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:41.944023 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:41.944282 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:41.944283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:41.944290 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:41.944546 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.944556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.945562 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.945565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.946739 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:41.946750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.947406 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.947409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:41.947542 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:41.947597 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:41.947937 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:41.947943 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:49:41.947995 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:49:41.948067 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":161,"threads_started":1}
I20260811 18:49:41.983837 29389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:41.984040 29389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:41.987215 29389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370eb7b725e7488baa9bb73503c5de5b: Sending DeleteTablet for 3 replicas of tablet 05d0a328cf2b4f5c9de63265b7e7ddc3
I20260811 18:49:41.988148 29706 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:43286
I20260811 18:49:41.988379 30004 tablet_replica.cc:330] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: stopping tablet replica
I20260811 18:49:41.988345 29875 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:40750
I20260811 18:49:41.988521 30004 raft_consensus.cc:2229] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:41.988667 30004 raft_consensus.cc:2258] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:41.988735 30006 tablet_replica.cc:330] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: stopping tablet replica
I20260811 18:49:41.988876 30006 raft_consensus.cc:2229] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:49:41.989322 30006 raft_consensus.cc:2258] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:41.990873 30004 ts_tablet_manager.cc:1905] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:41.995071 30006 ts_tablet_manager.cc:1905] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:41.998554 29546 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:41 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:45018
I20260811 18:49:41.998950 30005 tablet_replica.cc:330] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: stopping tablet replica
I20260811 18:49:41.999132 30005 raft_consensus.cc:2229] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:41.999292 30005 raft_consensus.cc:2258] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:42.000429 30005 ts_tablet_manager.cc:1905] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:42.004727 30006 ts_tablet_manager.cc:1918] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:42.004834 30005 ts_tablet_manager.cc:1918] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:42.005015 30005 log.cc:1196] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/05d0a328cf2b4f5c9de63265b7e7ddc3
I20260811 18:49:42.005036 30004 ts_tablet_manager.cc:1918] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:42.005234 30004 log.cc:1196] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/05d0a328cf2b4f5c9de63265b7e7ddc3
I20260811 18:49:42.005362 30006 log.cc:1196] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/05d0a328cf2b4f5c9de63265b7e7ddc3
I20260811 18:49:42.005522 30005 ts_tablet_manager.cc:1939] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P cd734af9c62b4f3888c25774efd0fe97: Deleting consensus metadata
I20260811 18:49:42.005883 30006 ts_tablet_manager.cc:1939] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 3c5a75e6bcd947858d8a31afd39607b0: Deleting consensus metadata
I20260811 18:49:42.006323 30004 ts_tablet_manager.cc:1939] T 05d0a328cf2b4f5c9de63265b7e7ddc3 P 5eb8573c017c426d8ba844025087b40d: Deleting consensus metadata
I20260811 18:49:42.006831 29374 catalog_manager.cc:4841] TS cd734af9c62b4f3888c25774efd0fe97 (127.27.115.193:43595): tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (table test-table [id=d89dc2b21ff24046af4bf22abcac75c8]) successfully deleted
I20260811 18:49:42.008057 29377 catalog_manager.cc:4841] TS 5eb8573c017c426d8ba844025087b40d (127.27.115.194:33109): tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (table test-table [id=d89dc2b21ff24046af4bf22abcac75c8]) successfully deleted
I20260811 18:49:42.009040 29378 catalog_manager.cc:4841] TS 3c5a75e6bcd947858d8a31afd39607b0 (127.27.115.195:39265): tablet 05d0a328cf2b4f5c9de63265b7e7ddc3 (table test-table [id=d89dc2b21ff24046af4bf22abcac75c8]) successfully deleted
Aug 11 18:49:42 dist-test-slave-9jjr krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:42.021330 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:41.990534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.27.115.193:43595)
0811 18:49:41.990669 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:41.990674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:41.994672 (+ 3998us) server_negotiation.cc:407] Connection header received
0811 18:49:41.994979 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:41.994984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:41.995075 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:41.995136 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:41.996804 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:41.997873 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:41.999977 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:42.000472 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:42.000602 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:42.014160 (+ 13558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:42.014207 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:42.014244 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:42.014328 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:42.017594 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:42.018555 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:42.018577 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:42.018579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:42.018772 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:42.019308 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:42.019312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:42.019314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:42.020030 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:42.020481 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:42.020663 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:49:42.020900 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 18:49:42.022867 29610 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45034
I20260811 18:49:42.043771 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:42.025718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53048 (local address 127.27.115.254:42613)
0811 18:49:42.025883 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:42.025888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:42.030646 (+ 4758us) server_negotiation.cc:407] Connection header received
0811 18:49:42.030830 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:42.030834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:42.030954 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:42.031134 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:42.032551 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:42.033507 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:42.035556 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:42.035977 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:42.036211 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:42.038646 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:42.038651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:42.038655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:42.038704 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:42.041062 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:42.041853 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:42.041858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:42.041860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:42.042015 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:42.042521 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:42.042525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:42.042527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:42.042826 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:42.042989 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:42.043374 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:49:42.043573 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 18:49:42.044641 29389 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53048
I20260811 18:49:42.047114 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 29475
I20260811 18:49:42.061229 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 29635
I20260811 18:49:42.077795 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 29801
I20260811 18:49:42.087742 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 29355
2026-08-11T18:49:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3050 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:42 dist-test-slave-9jjr krb5kdc[30058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:42 dist-test-slave-9jjr krb5kdc[30058](info): set up 2 sockets
Aug 11 18:49:42 dist-test-slave-9jjr krb5kdc[30058](info): commencing operation
krb5kdc: starting...
W20260811 18:49:43.238591 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:43 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:43.378551 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42255
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:43.497350 30071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:43.497659 30071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:43.497702 30071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:43.501785 30071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:43.501870 30071 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:49:43.501889 30071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:43.501909 30071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:43.501924 30071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:43.501940 30071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:43.509045 30071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:42255
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:43.510301 30071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:43.511420 30071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:43.520833 30077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:43.520898 30079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:43.520928 30076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:43.521404 30071 server_base.cc:1034] running on GCE node
I20260811 18:49:43.522312 30071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:43.523830 30071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:43.525076 30071 hybrid_clock.cc:648] HybridClock initialized: now 1786474183525041 us; error 58 us; skew 500 ppm
Aug 11 18:49:43 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:43.528403 30071 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:43.529662 30071 webserver.cc:460] Webserver started at http://127.27.115.254:35763/ using document root <none> and password file <none>
I20260811 18:49:43.530426 30071 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:43.530534 30071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:43.530820 30071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:43.533143 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "d4b6ff8b6984410a8251165e386c12bf"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "553fd160840c12607e7bac3786875c1d"
server_key_iv: "2d6975ad8f77ef542324ab6d293faf81"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.533778 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "d4b6ff8b6984410a8251165e386c12bf"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "553fd160840c12607e7bac3786875c1d"
server_key_iv: "2d6975ad8f77ef542324ab6d293faf81"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.538021 30071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:49:43.540956 30087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:43.542698 30071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:43.542860 30071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "d4b6ff8b6984410a8251165e386c12bf"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "553fd160840c12607e7bac3786875c1d"
server_key_iv: "2d6975ad8f77ef542324ab6d293faf81"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.542996 30071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:43.565193 30071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:43.568634 30071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:43.568864 30071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:43.577490 30071 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:42255
I20260811 18:49:43.577528 30156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:42255 every 8 connection(s)
I20260811 18:49:43.579171 30071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:43.582438 30157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:43.587093 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 30071
I20260811 18:49:43.587209 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:43.587507 28111 external_mini_cluster.cc:1428] Setting key 7f15fb4aae26384a5451861dacad7637
I20260811 18:49:43.589166 30157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Bootstrap starting.
I20260811 18:49:43.592252 30157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:43.593321 30157 log.cc:824] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:43.596292 30157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: No bootstrap required, opened a new log
Aug 11 18:49:43 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:43.601449 30157 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } }
I20260811 18:49:43.601807 30157 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:43.601863 30157 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b6ff8b6984410a8251165e386c12bf, State: Initialized, Role: FOLLOWER
I20260811 18:49:43.602488 30157 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } }
I20260811 18:49:43.602639 30157 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:43.602738 30157 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:43.602869 30157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:43.603926 30157 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } }
I20260811 18:49:43.604314 30157 leader_election.cc:302] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4b6ff8b6984410a8251165e386c12bf; no voters:
I20260811 18:49:43.604715 30157 leader_election.cc:288] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:43.604836 30162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:43.605058 30162 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [term 1 LEADER]: Becoming Leader. State: Replica: d4b6ff8b6984410a8251165e386c12bf, State: Running, Role: LEADER
I20260811 18:49:43.605394 30162 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } }
I20260811 18:49:43.605939 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:43.589172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.27.115.254:42255)
0811 18:49:43.589685 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:43.589706 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:43.589752 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:49:43.590759 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:43.590785 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:43.591240 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:43.591625 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:43.592516 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.594337 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.595672 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.596417 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.596594 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:43.599867 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:43.599875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:43.599900 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:43.599947 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:43.602221 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.602896 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.602902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.602908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.603119 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.603447 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.603450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.603452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.603920 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:43.604159 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:43.604472 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:49:43.604832 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":147,"threads_started":1}
I20260811 18:49:43.606096 30157 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:43.606801 30162 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b6ff8b6984410a8251165e386c12bf. Latest consensus state: current_term: 1 leader_uuid: "d4b6ff8b6984410a8251165e386c12bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } } }
I20260811 18:49:43.606923 30162 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:43.607326 30169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:43.607465 30163 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4b6ff8b6984410a8251165e386c12bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b6ff8b6984410a8251165e386c12bf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42255 } } }
I20260811 18:49:43.607664 30163 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:43.610661 30169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:43.616936 30169 catalog_manager.cc:1347] Generated new cluster ID: 28e724083d07452d95897b45488ed6c7
I20260811 18:49:43.617033 30169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:43.637888 30169 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:43.639416 30169 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:43.655745 30169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Generated new TSK 0
I20260811 18:49:43.656759 30169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:43.693681 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42255
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:43.827205 30189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:43.827587 30189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:43.827687 30189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:43.832202 30189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:43.832327 30189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:43.832413 30189 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:49:43.832458 30189 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:43.832531 30189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:43.839429 30189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:43.840816 30189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:43.842080 30189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:43.854735 30198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:43.854758 30196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:43.854745 30195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:43.855163 30189 server_base.cc:1034] running on GCE node
I20260811 18:49:43.855676 30189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:43.856475 30189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:43.857810 30189 hybrid_clock.cc:648] HybridClock initialized: now 1786474183857769 us; error 36 us; skew 500 ppm
Aug 11 18:49:43 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:43.861869 30189 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:43.863240 30189 webserver.cc:460] Webserver started at http://127.27.115.193:33599/ using document root <none> and password file <none>
I20260811 18:49:43.863984 30189 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:43.864070 30189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:43.864348 30189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:43.867233 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "29cdf73b73304a3ea4034ecf2e4e1201"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "8e67464c139b4695f1c8c978d979e923"
server_key_iv: "5c233995d25f7ac188b7b0cfbf1ecef9"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.868023 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "29cdf73b73304a3ea4034ecf2e4e1201"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "8e67464c139b4695f1c8c978d979e923"
server_key_iv: "5c233995d25f7ac188b7b0cfbf1ecef9"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.873495 30189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:49:43.876886 30206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:43.878917 30189 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:43.879143 30189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "29cdf73b73304a3ea4034ecf2e4e1201"
format_stamp: "Formatted at 2026-08-11 18:49:43 on dist-test-slave-9jjr"
server_key: "8e67464c139b4695f1c8c978d979e923"
server_key_iv: "5c233995d25f7ac188b7b0cfbf1ecef9"
server_key_version: "encryptionkey@0"
I20260811 18:49:43.879325 30189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:43.893575 30189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:43.898201 30189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:43.898486 30189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:43.900316 30189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:43.900411 30189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:43.900508 30189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:43.900566 30189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:49:43 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:43.913710 30189 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:40573
I20260811 18:49:43.914072 30339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:40573 every 8 connection(s)
I20260811 18:49:43.915093 30189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:43.917702 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:43.901663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:48841 (local address 127.27.115.254:42255)
0811 18:49:43.901823 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:43.901828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:43.903124 (+ 1296us) server_negotiation.cc:407] Connection header received
0811 18:49:43.904663 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:43.904669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:43.904759 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:43.904884 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:43.906703 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.907603 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.909366 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.909640 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.909785 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:43.913229 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:43.913233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:43.913236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:43.913267 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:43.915278 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.916005 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.916009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.916011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.916112 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.916553 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.916555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.916557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.916778 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:43.916860 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:43.917427 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 18:49:43.917558 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:49:43.918766 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:43.902276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.193:48841)
0811 18:49:43.902897 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:43.902946 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:43.904327 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:43.905129 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:43.905138 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:43.905621 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:43.906491 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:43.906520 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.907788 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.907792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:43.909190 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:43.909205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.909857 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.909861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:43.910080 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:43.910952 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:43.910972 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:43.912981 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:43.915430 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:43.915436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:43.915449 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:43.915857 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:43.916235 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:43.916239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:43.916248 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:43.916447 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:43.916882 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:43.916887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:43.917149 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 18:49:43.917341 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:49:43.917631 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":168,"threads_started":1}
I20260811 18:49:43.920641 30340 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42255
I20260811 18:49:43.921120 30340 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:43.921877 30340 heartbeater.cc:511] Master 127.27.115.254:42255 requested a full tablet report, sending...
I20260811 18:49:43.923751 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 30189
I20260811 18:49:43.923887 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:43.924083 30105 ts_manager.cc:194] Registered new tserver with Master: 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573)
I20260811 18:49:43.924250 28111 external_mini_cluster.cc:1428] Setting key a44d6c6639b16cbfdbe2e352f353c309
I20260811 18:49:43.925453 30105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:48841
I20260811 18:49:43.932641 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:43.923325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.193:56091)
0811 18:49:43.923763 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:43.923775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:43.923953 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:43.924398 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:43.924400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:43.924615 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:43.924783 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:43.924795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.925910 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.925913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:43.926705 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:43.926716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.927388 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.927392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:43.927484 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:43.928163 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:43.928180 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:43.928582 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:43.930772 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:43.930776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:43.930778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:43.930999 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:43.931411 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:43.931414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:43.931437 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:43.931545 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:43.932130 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:43.932132 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:43.932242 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 18:49:43.932329 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:49:43.932465 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":122,"threads_started":1}
I20260811 18:49:43.932888 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:43.923461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56091 (local address 127.27.115.254:42255)
0811 18:49:43.923606 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:43.923611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:43.923868 (+ 257us) server_negotiation.cc:407] Connection header received
0811 18:49:43.924093 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:43.924096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:43.924138 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:43.924235 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:43.924894 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.925650 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.926847 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:43.927242 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:43.927375 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:43.928752 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:43.928756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:43.928758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:43.928781 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:43.930621 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.931116 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.931123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.931128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.931299 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:43.931690 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:43.931697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:43.931702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:43.932013 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:43.932145 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:43.932444 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 18:49:43.932605 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:43.957674 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42255
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:44.084189 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:44.084528 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:44.084612 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:44.088927 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:44.089056 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:44.089141 30353 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:49:44.089308 30353 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:44.089461 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:44.095371 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:44.096582 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:44.097718 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:44.109691 30359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:44.109691 30362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:44.109711 30360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:44.110417 30353 server_base.cc:1034] running on GCE node
I20260811 18:49:44.110970 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:44.111730 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:44.113027 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786474184112999 us; error 52 us; skew 500 ppm
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:44.117130 30353 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:44.119035 30353 webserver.cc:460] Webserver started at http://127.27.115.194:41277/ using document root <none> and password file <none>
I20260811 18:49:44.119774 30353 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:44.119894 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:44.120157 30353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:44.123056 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "cb9646eec87a1987c1fbd93abc9a17b4"
server_key_iv: "1a0ff8a6644a8ee644e127a2644edf6d"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.123809 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "cb9646eec87a1987c1fbd93abc9a17b4"
server_key_iv: "1a0ff8a6644a8ee644e127a2644edf6d"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.129101 30353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:49:44.132936 30369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:44.134743 30353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:49:44.134938 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "cb9646eec87a1987c1fbd93abc9a17b4"
server_key_iv: "1a0ff8a6644a8ee644e127a2644edf6d"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.135109 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:44.153275 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:44.158018 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:44.158330 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:44.160194 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:44.160286 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:44.160375 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:44.160422 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:44.173894 30353 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:45417
I20260811 18:49:44.173960 30509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:45417 every 8 connection(s)
I20260811 18:49:44.175071 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:44.177053 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.161339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:44615 (local address 127.27.115.254:42255)
0811 18:49:44.161553 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.161560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.162600 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 18:49:44.163768 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.163772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.163876 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.163948 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.165703 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.166576 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.167889 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.168152 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.168268 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.172081 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.172086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.172089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.172163 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.174270 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.174970 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.174975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.174977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.175090 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.175512 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.175516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.175518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.175821 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.175932 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.176599 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.176734 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:49:44.177168 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 30353
I20260811 18:49:44.177266 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:44.177558 28111 external_mini_cluster.cc:1428] Setting key e1bc6cc4e25033adebd1f31096b03d9e
I20260811 18:49:44.177848 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.161802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.194:44615)
0811 18:49:44.162400 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.162459 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.163459 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.164484 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.164492 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.164970 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:44.165520 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.165535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.166758 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.166762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.167732 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.167744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.168359 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.168364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.168622 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.169420 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:44.169450 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:44.171818 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:44.174450 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.174460 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.174480 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.174832 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.175214 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.175218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.175220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.175396 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.175997 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:44.176004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:44.176302 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.176487 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.176788 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":158,"threads_started":1}
I20260811 18:49:44.181445 30511 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42255
I20260811 18:49:44.181674 30511 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:44.182253 30511 heartbeater.cc:511] Master 127.27.115.254:42255 requested a full tablet report, sending...
I20260811 18:49:44.183477 30104 ts_manager.cc:194] Registered new tserver with Master: 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417)
I20260811 18:49:44.184267 30104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:44615
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:49:44.191972 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.182305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:39919 (local address 127.27.115.254:42255)
0811 18:49:44.182467 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.182472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.182625 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:49:44.182761 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.182765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.182820 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.182923 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.183595 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.184472 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.185969 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.186401 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.186504 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.188107 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.188111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.188114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.188150 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.190183 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.190688 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.190693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.190695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.190804 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.191106 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.191109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.191110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.191361 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.191446 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.191715 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.191837 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:49:44.192128 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.182190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.194:39919)
0811 18:49:44.182512 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.182531 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.182655 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.182961 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.182966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.183271 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:44.183431 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.183441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.184688 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.184694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.185794 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.185811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.186668 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.186670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.186743 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.187540 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:44.187553 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:44.187927 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:44.190308 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.190310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.190312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.190541 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.190919 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.190922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.190923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.191017 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.191450 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:44.191452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:44.191544 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.191628 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.191852 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":4,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:44.213385 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:42255
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:44.333649 30520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:44.334033 30520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:44.334158 30520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:44.338593 30520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:44.338732 30520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:44.338812 30520 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:49:44.338846 30520 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:49:44.338924 30520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:44.345023 30520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:42255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:44.346387 30520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:44.347630 30520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:44.358492 30529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:44.358477 30526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:44.358597 30520 server_base.cc:1034] running on GCE node
W20260811 18:49:44.358506 30527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:44.359229 30520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:44.359977 30520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:44.361217 30520 hybrid_clock.cc:648] HybridClock initialized: now 1786474184361182 us; error 48 us; skew 500 ppm
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:44.364204 30520 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:44.365185 30520 webserver.cc:460] Webserver started at http://127.27.115.195:39699/ using document root <none> and password file <none>
I20260811 18:49:44.365742 30520 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:44.365803 30520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:44.365983 30520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:44.368168 30520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "8c09826cd05345aaa05ab46e8029480f"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "2a381df31c05c85164a434cffe88a3f5"
server_key_iv: "672f0fde29f2b67cefb190fb66ccf952"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.368734 30520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "8c09826cd05345aaa05ab46e8029480f"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "2a381df31c05c85164a434cffe88a3f5"
server_key_iv: "672f0fde29f2b67cefb190fb66ccf952"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.372784 30520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 18:49:44.375322 30538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:44.376766 30520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:49:44.376880 30520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "8c09826cd05345aaa05ab46e8029480f"
format_stamp: "Formatted at 2026-08-11 18:49:44 on dist-test-slave-9jjr"
server_key: "2a381df31c05c85164a434cffe88a3f5"
server_key_iv: "672f0fde29f2b67cefb190fb66ccf952"
server_key_version: "encryptionkey@0"
I20260811 18:49:44.377036 30520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:44.406800 30520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:44.410774 30520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:44.410974 30520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:44.412611 30520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:44.412676 30520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:44.412750 30520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:44.412782 30520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:44.424096 30520 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:37877
I20260811 18:49:44.424185 30667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:37877 every 8 connection(s)
I20260811 18:49:44.425262 30520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:44.428238 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.413684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:52035 (local address 127.27.115.254:42255)
0811 18:49:44.413828 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.413832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.415130 (+ 1298us) server_negotiation.cc:407] Connection header received
0811 18:49:44.416647 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.416659 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.416742 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.416860 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.418602 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.419168 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.420726 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.420984 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.421049 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.424207 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.424213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.424216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.424248 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.425964 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.426634 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.426637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.426639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.426734 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.427096 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.427099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.427101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.427299 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.427411 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.427919 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.428057 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:49:44.429044 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.414352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.195:52035)
0811 18:49:44.414926 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.414977 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.416254 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.417042 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.417050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.417647 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:44.418437 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.418459 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.419378 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.419384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.420478 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.420489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.421247 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.421250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.421397 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.422106 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:44.422131 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:44.423951 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:44.426098 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.426106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.426122 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.426507 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.426834 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.426838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.426840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.426989 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.427411 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:44.427418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:44.427671 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.427861 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.428208 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":174,"threads_started":1}
I20260811 18:49:44.430292 30668 heartbeater.cc:347] Connected to a master server at 127.27.115.254:42255
I20260811 18:49:44.430542 30668 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:44.431187 30668 heartbeater.cc:511] Master 127.27.115.254:42255 requested a full tablet report, sending...
I20260811 18:49:44.432260 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 30520
I20260811 18:49:44.432391 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:44.432504 30105 ts_manager.cc:194] Registered new tserver with Master: 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:44.432730 28111 external_mini_cluster.cc:1428] Setting key 001237d9362fe27b4e8e1ee5d4a289df
I20260811 18:49:44.433305 30105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:52035
I20260811 18:49:44.435510 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:49:44.441004 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.433182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:44423 (local address 127.27.115.254:42255)
0811 18:49:44.433668 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.433673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.433689 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:49:44.433725 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.433727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.433770 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.433832 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.434399 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.434988 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.435917 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.436151 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.436274 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.437474 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.437478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.437481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.437507 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.439240 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.439680 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.439686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.439690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.439783 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.440073 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.440077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.440079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.440333 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.440443 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.440716 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.440830 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402}
I20260811 18:49:44.441017 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.433120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.27.115.195:44423)
0811 18:49:44.433518 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.433531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.433610 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.433878 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.433881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.434151 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:44.434278 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.434285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.435123 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.435126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.435791 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.435801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.436279 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.436283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.436375 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.436972 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:44.436984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:44.437322 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:44.439358 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.439361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.439364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.439575 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.439884 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.439887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.439889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.439977 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.440433 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:44.440436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:44.440543 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.440655 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.440828 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":3,"thread_start_us":156,"threads_started":1}
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:44.464468 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.455552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.27.115.254:42255)
0811 18:49:44.455694 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.455698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.455712 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:49:44.455767 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.455770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.455811 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.455887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.456648 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.457218 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.458230 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.458564 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.458690 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.461511 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.461514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.461516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.461545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.462939 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.463372 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.463375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.463376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.463459 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.463710 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.463712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.463713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.463894 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.463997 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.464237 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.464340 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:49:44.467801 30104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:44.471087 30104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:44.486627 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.480451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.27.115.194:45417)
0811 18:49:44.480863 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.480875 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.480901 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:49:44.481152 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.481158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.481469 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.481643 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:44.482253 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.483216 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.484873 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.485747 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.485889 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.486028 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.486313 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.486426 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":106,"threads_started":1}
I20260811 18:49:44.486606 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.480313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45417 (local address 127.0.0.1:34504)
0811 18:49:44.480729 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.480790 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.480924 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.481642 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.481648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.481681 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:44.482107 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.482114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.483367 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.483373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.484733 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.484745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.485893 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.485898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.486011 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.486064 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.486266 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.486420 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"spinlock_wait_cycles":6912,"thread_start_us":108,"threads_started":1}
I20260811 18:49:44.487040 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.480679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:37877 (local address 127.0.0.1:46284)
0811 18:49:44.481307 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.481323 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.481415 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.482673 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.482676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.482699 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:44.482781 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.482787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.484274 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.484277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.485479 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.485491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.486648 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.486655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.486749 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.486772 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.486825 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.486897 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260811 18:49:44.487350 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.481697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.27.115.193:40573)
0811 18:49:44.481949 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.481956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.481979 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:49:44.482060 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.482065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.482300 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.482438 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:44.482652 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.483785 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.485272 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.485829 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.485958 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.486098 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.487113 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.487225 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260811 18:49:44.486608 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.481136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.0.0.1:32794)
0811 18:49:44.481855 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.481870 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.481966 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.482434 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.482437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.482457 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:44.482548 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.482554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.483969 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.483972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.485153 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.485162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.486187 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.486193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.486303 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.486325 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.486396 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.486446 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":32,"spinlock_wait_cycles":896,"thread_start_us":98,"threads_started":1}
I20260811 18:49:44.487819 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.481209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.27.115.195:37877)
0811 18:49:44.482173 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.482184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.482204 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:49:44.482271 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.482277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.482434 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.482926 (+ 492us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:44.483093 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.484154 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.485651 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.486488 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.487362 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.487495 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.487546 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.487677 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":125,"threads_started":1}
I20260811 18:49:44.489914 30434 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (DEFAULT_TABLE table=test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:44.490301 30261 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (DEFAULT_TABLE table=test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:44.490305 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (DEFAULT_TABLE table=test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:44.491093 30434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.491317 30261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.491325 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.499015 30698 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Bootstrap starting.
I20260811 18:49:44.499920 30699 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Bootstrap starting.
I20260811 18:49:44.500451 30700 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Bootstrap starting.
I20260811 18:49:44.501590 30698 tablet_bootstrap.cc:654] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.502060 30699 tablet_bootstrap.cc:654] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.502485 30698 log.cc:824] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:44.502523 30700 tablet_bootstrap.cc:654] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.502942 30699 log.cc:824] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:44.503350 30700 log.cc:824] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:44.505025 30699 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: No bootstrap required, opened a new log
I20260811 18:49:44.505096 30698 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: No bootstrap required, opened a new log
I20260811 18:49:44.505293 30699 ts_tablet_manager.cc:1397] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 18:49:44.505296 30698 ts_tablet_manager.cc:1397] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:49:44.507987 30700 tablet_bootstrap.cc:492] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: No bootstrap required, opened a new log
I20260811 18:49:44.508248 30700 ts_tablet_manager.cc:1397] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 18:49:44.509446 30699 raft_consensus.cc:348] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.509510 30698 raft_consensus.cc:348] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.509745 30699 raft_consensus.cc:374] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.509796 30699 raft_consensus.cc:729] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.509804 30698 raft_consensus.cc:374] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.509859 30698 raft_consensus.cc:729] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.510332 30699 consensus_queue.cc:260] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.510373 30698 consensus_queue.cc:260] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.511459 30699 ts_tablet_manager.cc:1428] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:49:44.511461 30698 ts_tablet_manager.cc:1428] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:49:44.511865 30340 heartbeater.cc:503] Master 127.27.115.254:42255 was elected leader, sending a full tablet report...
I20260811 18:49:44.512080 30511 heartbeater.cc:503] Master 127.27.115.254:42255 was elected leader, sending a full tablet report...
I20260811 18:49:44.512562 30700 raft_consensus.cc:348] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.512780 30700 raft_consensus.cc:374] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.512811 30700 raft_consensus.cc:729] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.513296 30700 consensus_queue.cc:260] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.514304 30668 heartbeater.cc:503] Master 127.27.115.254:42255 was elected leader, sending a full tablet report...
I20260811 18:49:44.514432 30700 ts_tablet_manager.cc:1428] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260811 18:49:44.666837 30342 tablet.cc:2369] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:44.676620 30670 tablet.cc:2369] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:44.676612 30512 tablet.cc:2369] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:49:44.694304 30704 raft_consensus.cc:482] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:44.694583 30704 raft_consensus.cc:504] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.696094 30704 leader_election.cc:288] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:44.700855 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.696505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:57905 (local address 127.27.115.194:45417)
0811 18:49:44.696666 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.696671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.696986 (+ 315us) server_negotiation.cc:407] Connection header received
0811 18:49:44.697159 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.697163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.697217 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.697311 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:44.697634 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.698266 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.699732 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.700190 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.700346 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.700410 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.700611 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.700695 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:49:44.700896 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.696442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45417 (local address 127.27.115.193:57905)
0811 18:49:44.696870 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.696882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.697030 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.697327 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.697330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.697348 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:44.697506 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.697512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.698399 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.698408 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.699526 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.699540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.700346 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.700350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.700456 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.700490 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.700591 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.700700 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":51,"spinlock_wait_cycles":1408,"thread_start_us":142,"threads_started":1}
I20260811 18:49:44.701078 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.696597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60919 (local address 127.27.115.195:37877)
0811 18:49:44.696812 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.696818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.696935 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:49:44.697141 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.697144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.697207 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.697296 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:44.697624 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.698391 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.699891 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.700411 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.700557 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.700615 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.700827 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.700903 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 18:49:44.701493 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.696428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:37877 (local address 127.27.115.193:60919)
0811 18:49:44.696812 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.696829 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.697004 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.697317 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.697320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.697338 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:44.697498 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.697504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.698529 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.698534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.699727 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.699741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.700557 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.700561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.700688 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.700715 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.700806 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.700887 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":89,"spinlock_wait_cycles":15616,"thread_start_us":134,"threads_started":1}
I20260811 18:49:44.701865 30455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9e4eca62734d4d9cf2c16dcfaa3177" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" is_pre_election: true
I20260811 18:49:44.702248 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9e4eca62734d4d9cf2c16dcfaa3177" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f" is_pre_election: true
I20260811 18:49:44.702350 30455 raft_consensus.cc:2454] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:44.702558 30615 raft_consensus.cc:2454] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:44.703003 30208 leader_election.cc:302] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 29cdf73b73304a3ea4034ecf2e4e1201; no voters:
I20260811 18:49:44.703404 30704 raft_consensus.cc:2781] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:44.703502 30704 raft_consensus.cc:482] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:44.703536 30704 raft_consensus.cc:3037] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.704774 30704 raft_consensus.cc:504] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.705240 30704 leader_election.cc:288] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Requested vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:44.705808 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9e4eca62734d4d9cf2c16dcfaa3177" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f"
I20260811 18:49:44.705978 30615 raft_consensus.cc:3037] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.705888 30455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9e4eca62734d4d9cf2c16dcfaa3177" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
I20260811 18:49:44.706212 30455 raft_consensus.cc:3037] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.707593 30455 raft_consensus.cc:2454] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:44.708114 30615 raft_consensus.cc:2454] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:44.708253 30208 leader_election.cc:302] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 29cdf73b73304a3ea4034ecf2e4e1201; no voters:
I20260811 18:49:44.708568 30704 raft_consensus.cc:2781] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:44.708856 30704 raft_consensus.cc:686] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 LEADER]: Becoming Leader. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Running, Role: LEADER
I20260811 18:49:44.709301 30704 consensus_queue.cc:237] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.713608 30104 catalog_manager.cc:5495] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193). New cstate: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:49:44 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:44.761716 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.752311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.27.115.254:42255)
0811 18:49:44.752450 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.752454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.752747 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:49:44.752843 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.752846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.752893 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.753053 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:44.753775 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.754400 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.755207 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.755419 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.755528 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.757927 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:44.757930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:44.757933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:44.757955 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:44.759664 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.760253 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.760257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.760258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.760334 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:44.760619 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:44.760625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:44.760627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:44.760840 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:44.760957 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.761291 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.761503 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":53}
I20260811 18:49:44.761724 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.752214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42255 (local address 127.0.0.1:39954)
0811 18:49:44.752625 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.752644 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.752758 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.753089 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.753092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.753540 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:44.753645 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.753653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.754509 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.754515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.755073 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.755082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.755538 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.755541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.755663 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.756233 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:44.756245 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:44.757751 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:44.759827 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.759840 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.759850 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.760119 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.760436 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:44.760439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:44.760440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:44.760526 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:44.760959 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:44.760970 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:44.761100 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.761234 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.761573 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":144,"thread_start_us":180,"threads_started":1}
I20260811 18:49:44.767185 30105 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:39954:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:49:44.768144 30105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:44.774706 30434 tablet_service.cc:1467] Processing CreateTablet for tablet 6d272a074f7c460da6b7f70e4b7f68ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d9669af44b746818ef856e67902a406]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:44.774813 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 6d272a074f7c460da6b7f70e4b7f68ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d9669af44b746818ef856e67902a406]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:44.774693 30261 tablet_service.cc:1467] Processing CreateTablet for tablet 6d272a074f7c460da6b7f70e4b7f68ad (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d9669af44b746818ef856e67902a406]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:49:44.775565 30434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d272a074f7c460da6b7f70e4b7f68ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.775565 30261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d272a074f7c460da6b7f70e4b7f68ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.775565 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d272a074f7c460da6b7f70e4b7f68ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:44.778692 30698 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: Bootstrap starting.
I20260811 18:49:44.778982 30700 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: Bootstrap starting.
I20260811 18:49:44.778982 30699 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: Bootstrap starting.
I20260811 18:49:44.780162 30698 tablet_bootstrap.cc:654] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.780167 30699 tablet_bootstrap.cc:654] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.780167 30700 tablet_bootstrap.cc:654] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:44.781873 30698 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: No bootstrap required, opened a new log
I20260811 18:49:44.781999 30698 ts_tablet_manager.cc:1397] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:49:44.781998 30700 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: No bootstrap required, opened a new log
I20260811 18:49:44.782095 30700 ts_tablet_manager.cc:1397] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:49:44.782423 30699 tablet_bootstrap.cc:492] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: No bootstrap required, opened a new log
I20260811 18:49:44.782523 30699 ts_tablet_manager.cc:1397] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:49:44.782639 30698 raft_consensus.cc:348] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.782678 30700 raft_consensus.cc:348] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.782781 30698 raft_consensus.cc:374] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.782815 30698 raft_consensus.cc:729] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.782809 30700 raft_consensus.cc:374] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.782855 30700 raft_consensus.cc:729] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.782958 30698 consensus_queue.cc:260] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.783010 30700 consensus_queue.cc:260] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.783051 30699 raft_consensus.cc:348] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.783186 30699 raft_consensus.cc:374] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:44.783226 30699 raft_consensus.cc:729] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Initialized, Role: FOLLOWER
I20260811 18:49:44.783319 30698 ts_tablet_manager.cc:1428] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:49:44.783366 30700 ts_tablet_manager.cc:1428] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:44.783404 30699 consensus_queue.cc:260] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.783675 30704 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.783648 30705 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.783818 30704 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.783818 30705 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.784076 30699 ts_tablet_manager.cc:1428] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:49:44.784051 30707 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.784230 30707 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.853916 30704 raft_consensus.cc:482] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:44.854087 30704 raft_consensus.cc:504] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.855100 30704 leader_election.cc:288] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:44.855947 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d272a074f7c460da6b7f70e4b7f68ad" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f" is_pre_election: true
I20260811 18:49:44.855959 30455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d272a074f7c460da6b7f70e4b7f68ad" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" is_pre_election: true
I20260811 18:49:44.856153 30615 raft_consensus.cc:2454] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:44.856186 30455 raft_consensus.cc:2454] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:44.856629 30209 leader_election.cc:302] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cdf73b73304a3ea4034ecf2e4e1201, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:44.856946 30704 raft_consensus.cc:2781] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:44.857021 30704 raft_consensus.cc:482] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:44.857124 30704 raft_consensus.cc:3037] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.858566 30704 raft_consensus.cc:504] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.859102 30704 leader_election.cc:288] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Requested vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:44.859820 30455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d272a074f7c460da6b7f70e4b7f68ad" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
I20260811 18:49:44.860034 30455 raft_consensus.cc:3037] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.860113 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d272a074f7c460da6b7f70e4b7f68ad" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f"
I20260811 18:49:44.860324 30615 raft_consensus.cc:3037] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:44.861384 30455 raft_consensus.cc:2454] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:44.861625 30615 raft_consensus.cc:2454] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:44.861970 30208 leader_election.cc:302] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 29cdf73b73304a3ea4034ecf2e4e1201; no voters:
I20260811 18:49:44.862291 30704 raft_consensus.cc:2781] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:44.862512 30704 raft_consensus.cc:686] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 LEADER]: Becoming Leader. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Running, Role: LEADER
I20260811 18:49:44.862699 30704 consensus_queue.cc:237] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:44.864146 30710 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: TxnStatusTablet state changed. Reason: New leader 29cdf73b73304a3ea4034ecf2e4e1201. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.864295 30710 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:44.864835 30728 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:49:44.865144 30728 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:49:44.866037 30105 catalog_manager.cc:5495] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193). New cstate: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } health_report { overall_health: UNKNOWN } } }
I20260811 18:49:44.896536 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.891416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.0.0.1:32806)
0811 18:49:44.891739 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.891753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.891865 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.892344 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.892347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.892355 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:44.892476 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.892481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.893643 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.893646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.894411 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.894423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.895074 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.895077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.895174 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.895231 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:44.896205 (+ 974us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:44.896225 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.896291 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.896383 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 18:49:44.896559 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.891556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.27.115.193:40573)
0811 18:49:44.891729 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.891735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.891803 (+ 68us) server_negotiation.cc:407] Connection header received
0811 18:49:44.892026 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.892030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.892172 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.892303 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:44.892603 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.893487 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.894550 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.894941 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.895066 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.895421 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:44.896065 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:44.896212 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.896315 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.896393 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:49:44.903514 30455 raft_consensus.cc:1261] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:44.903514 30615 raft_consensus.cc:1261] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:44.904488 30710 consensus_queue.cc:1035] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:44.904762 30704 consensus_queue.cc:1035] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:44.910208 30705 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: TxnStatusTablet state changed. Reason: New leader 29cdf73b73304a3ea4034ecf2e4e1201. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.910369 30705 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.917165 30707 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: TxnStatusTablet state changed. Reason: New leader 29cdf73b73304a3ea4034ecf2e4e1201. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.917346 30707 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.918550 30710 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.918596 30704 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.918705 30710 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:44.918709 30704 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 29cdf73b73304a3ea4034ecf2e4e1201: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:49:44.920059 30731 mvcc.cc:205] Tried to move back new op lower bound from 7317398261355900928 to 7317398261199749120. Current Snapshot: MvccSnapshot[applied={T|T < 7317398261355900928}]
I20260811 18:49:44.921129 30705 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.921353 30705 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 1b6c6c24ac3e4b818cf22ddaac708d76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.922619 30707 tablet_replica.cc:439] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } } }
I20260811 18:49:44.922782 30707 tablet_replica.cc:442] T 6d272a074f7c460da6b7f70e4b7f68ad P 8c09826cd05345aaa05ab46e8029480f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:49:44.927047 30735 mvcc.cc:205] Tried to move back new op lower bound from 7317398261355900928 to 7317398261199749120. Current Snapshot: MvccSnapshot[applied={T|T < 7317398261355900928 or (T in {7317398261355900928})}]
I20260811 18:49:44.935302 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.930697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.27.115.193:40573)
0811 18:49:44.930854 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.930859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.930934 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:49:44.931110 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.931114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.931175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.931268 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:44.931600 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.932432 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.933882 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.934286 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.934401 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.934742 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:44.934856 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:44.934930 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.935090 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.935158 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:49:44.949982 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.945157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:47259 (local address 127.27.115.193:40573)
0811 18:49:44.945298 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:44.945303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:44.945590 (+ 287us) server_negotiation.cc:407] Connection header received
0811 18:49:44.945754 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:44.945757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:44.945813 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:44.945904 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:44.946215 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.947011 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.948573 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.948918 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.949044 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.949402 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:44.949507 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:44.949590 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:44.949755 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 18:49:44.949816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:49:44.949991 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:44.945104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.27.115.193:47259)
0811 18:49:44.945479 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:44.945497 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:44.945655 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:44.945927 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:44.945929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:44.945935 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:44.946070 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.946075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.947152 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.947155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.948423 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:44.948439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:44.949064 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:44.949066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:44.949165 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:44.949226 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:44.949637 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:44.949645 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:49:44.949720 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:49:44.949808 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":149,"threads_started":1}
I20260811 18:49:44.958590 30455 raft_consensus.cc:1261] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:44.958609 30615 raft_consensus.cc:1261] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:44.959074 30710 consensus_queue.cc:1035] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:44.959288 30738 consensus_queue.cc:1035] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:44.965436 30732 mvcc.cc:205] Tried to move back new op lower bound from 7317398261586493440 to 7317398260572196864. Current Snapshot: MvccSnapshot[applied={T|T < 7317398261586493440}]
I20260811 18:49:45.009866 30105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:45.010098 30105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:45.012959 30105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Sending DeleteTablet for 3 replicas of tablet 3f9e4eca62734d4d9cf2c16dcfaa3177
I20260811 18:49:45.014004 30260 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:45 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:32794
I20260811 18:49:45.014047 30434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:45 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34504
I20260811 18:49:45.013999 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:45 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46284
I20260811 18:49:45.014626 30751 tablet_replica.cc:330] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: stopping tablet replica
I20260811 18:49:45.014762 30749 tablet_replica.cc:330] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: stopping tablet replica
I20260811 18:49:45.015005 30751 raft_consensus.cc:2229] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:45.015004 30749 raft_consensus.cc:2229] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:45.015239 30749 raft_consensus.cc:2258] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:45.015250 30751 raft_consensus.cc:2258] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:45.015594 30750 tablet_replica.cc:330] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: stopping tablet replica
I20260811 18:49:45.015717 30105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:49:45.015895 30750 raft_consensus.cc:2229] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:49:45.016240 30105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:45.016294 30750 raft_consensus.cc:2258] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:45.016764 30749 ts_tablet_manager.cc:1905] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:45.019532 30751 ts_tablet_manager.cc:1905] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:45.020450 30749 ts_tablet_manager.cc:1918] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:45.020578 30749 log.cc:1196] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/3f9e4eca62734d4d9cf2c16dcfaa3177
I20260811 18:49:45.020953 30749 ts_tablet_manager.cc:1939] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 8c09826cd05345aaa05ab46e8029480f: Deleting consensus metadata
I20260811 18:49:45.022617 30090 catalog_manager.cc:4841] TS 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877): tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (table test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]) successfully deleted
I20260811 18:49:45.022948 30593 tablet_service.cc:1467] Processing CreateTablet for tablet d7f3803ee0264ce1820333157cd07354 (DEFAULT_TABLE table=test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:45.023039 30260 tablet_service.cc:1467] Processing CreateTablet for tablet d7f3803ee0264ce1820333157cd07354 (DEFAULT_TABLE table=test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:45.023303 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f3803ee0264ce1820333157cd07354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:45.023186 30434 tablet_service.cc:1467] Processing CreateTablet for tablet d7f3803ee0264ce1820333157cd07354 (DEFAULT_TABLE table=test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:45.023379 30260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f3803ee0264ce1820333157cd07354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:45.023569 30434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f3803ee0264ce1820333157cd07354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:45.026196 30700 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Bootstrap starting.
I20260811 18:49:45.026196 30699 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Bootstrap starting.
I20260811 18:49:45.027293 30699 tablet_bootstrap.cc:654] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:45.027294 30700 tablet_bootstrap.cc:654] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:45.028249 30698 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Bootstrap starting.
I20260811 18:49:45.029387 30698 tablet_bootstrap.cc:654] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:45.031255 30751 ts_tablet_manager.cc:1918] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:45.031392 30751 log.cc:1196] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/3f9e4eca62734d4d9cf2c16dcfaa3177
I20260811 18:49:45.031744 30751 ts_tablet_manager.cc:1939] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting consensus metadata
I20260811 18:49:45.031716 30750 ts_tablet_manager.cc:1905] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:45.033414 30089 catalog_manager.cc:4841] TS 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417): tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (table test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]) successfully deleted
I20260811 18:49:45.034782 30698 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: No bootstrap required, opened a new log
I20260811 18:49:45.034898 30698 ts_tablet_manager.cc:1397] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:49:45.034904 30750 ts_tablet_manager.cc:1918] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:45.034993 30750 log.cc:1196] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/3f9e4eca62734d4d9cf2c16dcfaa3177
I20260811 18:49:45.035274 30750 ts_tablet_manager.cc:1939] T 3f9e4eca62734d4d9cf2c16dcfaa3177 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting consensus metadata
I20260811 18:49:45.035498 30698 raft_consensus.cc:348] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.035640 30698 raft_consensus.cc:374] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:45.035678 30698 raft_consensus.cc:729] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Initialized, Role: FOLLOWER
I20260811 18:49:45.035845 30698 consensus_queue.cc:260] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.036185 30698 ts_tablet_manager.cc:1428] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:49:45.036733 30699 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: No bootstrap required, opened a new log
I20260811 18:49:45.036849 30699 ts_tablet_manager.cc:1397] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:49:45.037081 30700 tablet_bootstrap.cc:492] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: No bootstrap required, opened a new log
I20260811 18:49:45.037171 30700 ts_tablet_manager.cc:1397] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:49:45.037390 30699 raft_consensus.cc:348] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.037508 30699 raft_consensus.cc:374] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:45.037542 30699 raft_consensus.cc:729] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Initialized, Role: FOLLOWER
I20260811 18:49:45.037681 30699 consensus_queue.cc:260] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.037660 30700 raft_consensus.cc:348] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.037801 30700 raft_consensus.cc:374] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:45.037835 30700 raft_consensus.cc:729] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Initialized, Role: FOLLOWER
I20260811 18:49:45.037976 30699 ts_tablet_manager.cc:1428] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:49:45.037962 30700 consensus_queue.cc:260] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.038302 30700 ts_tablet_manager.cc:1428] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:49:45.039682 30089 catalog_manager.cc:4841] TS 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573): tablet 3f9e4eca62734d4d9cf2c16dcfaa3177 (table test-table [id=c4e27c9dca3d447d8808a45cbe861f5c]) successfully deleted
I20260811 18:49:45.040046 30705 raft_consensus.cc:482] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:45.040158 30705 raft_consensus.cc:504] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.041621 30705 leader_election.cc:288] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:45.046648 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.042071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:37877 (local address 127.27.115.194:35525)
0811 18:49:45.042485 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:45.042498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:45.042639 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:45.042986 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:45.042990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:45.043013 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:45.043164 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.043172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.044275 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.044278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.045417 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.045427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.046104 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.046108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.046268 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.046296 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:49:45.046398 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:49:45.046475 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1,"thread_start_us":169,"threads_started":1}
I20260811 18:49:45.046622 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.042275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:35525 (local address 127.27.115.195:37877)
0811 18:49:45.042422 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:45.042427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:45.042566 (+ 139us) server_negotiation.cc:407] Connection header received
0811 18:49:45.042794 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:45.042797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:45.042850 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:45.042945 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:45.043321 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.043991 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.045548 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.045955 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.046103 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.046236 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:45.046419 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:49:45.046492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:49:45.047541 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f3803ee0264ce1820333157cd07354" candidate_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f" is_pre_election: true
I20260811 18:49:45.047721 30615 raft_consensus.cc:2454] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6c6c24ac3e4b818cf22ddaac708d76 in term 0.
I20260811 18:49:45.048242 30372 leader_election.cc:302] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:45.048336 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.042746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:38873 (local address 127.27.115.193:40573)
0811 18:49:45.043145 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:45.043150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:45.043163 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:49:45.043210 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:45.043213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:45.043256 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:45.043330 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:45.044122 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.044865 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.046549 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.047214 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.047318 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.047369 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:45.048118 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 18:49:45.048209 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260811 18:49:45.048604 30705 raft_consensus.cc:2781] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:45.048681 30705 raft_consensus.cc:482] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:45.048749 30705 raft_consensus.cc:3037] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:45.048776 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.041874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.27.115.194:38873)
0811 18:49:45.042346 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:45.042362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:45.042471 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:45.043819 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:45.043823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:45.043842 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:45.043964 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.043970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.045014 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.045018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.046383 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.046396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.047513 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.047517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.047624 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.047646 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:49:45.048534 (+ 888us) client_negotiation.cc:240] Negotiation successful
0811 18:49:45.048619 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":124,"threads_started":1}
I20260811 18:49:45.049340 30285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f3803ee0264ce1820333157cd07354" candidate_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" is_pre_election: true
I20260811 18:49:45.049598 30285 raft_consensus.cc:2454] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6c6c24ac3e4b818cf22ddaac708d76 in term 0.
I20260811 18:49:45.050196 30705 raft_consensus.cc:504] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.050575 30705 leader_election.cc:288] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [CANDIDATE]: Term 1 election: Requested vote from peers 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:45.050926 30285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f3803ee0264ce1820333157cd07354" candidate_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cdf73b73304a3ea4034ecf2e4e1201"
I20260811 18:49:45.050935 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f3803ee0264ce1820333157cd07354" candidate_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f"
I20260811 18:49:45.051059 30615 raft_consensus.cc:3037] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:45.051059 30285 raft_consensus.cc:3037] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:45.052289 30615 raft_consensus.cc:2454] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6c6c24ac3e4b818cf22ddaac708d76 in term 1.
I20260811 18:49:45.052289 30285 raft_consensus.cc:2454] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6c6c24ac3e4b818cf22ddaac708d76 in term 1.
I20260811 18:49:45.052647 30371 leader_election.cc:302] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 29cdf73b73304a3ea4034ecf2e4e1201, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:45.052843 30705 raft_consensus.cc:2781] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:45.053102 30705 raft_consensus.cc:686] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 LEADER]: Becoming Leader. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Running, Role: LEADER
I20260811 18:49:45.053370 30705 consensus_queue.cc:237] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:45.055980 30106 catalog_manager.cc:5495] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } health_report { overall_health: UNKNOWN } } }
I20260811 18:49:45.074688 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.070237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.27.115.194:45417)
0811 18:49:45.070378 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:45.070383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:45.070441 (+ 58us) server_negotiation.cc:407] Connection header received
0811 18:49:45.070618 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:45.070624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:45.070699 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:45.070822 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:45.071032 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.071866 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.072692 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.073076 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.073197 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.073585 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:45.074256 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:45.074376 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:45.074457 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 18:49:45.074523 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:49:45.089251 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.084122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.27.115.194:49475)
0811 18:49:45.084664 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:45.084677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:45.084798 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:45.085094 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:45.085096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:45.085104 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:45.085233 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.085238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.086393 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.086396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.087531 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.087545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.088299 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.088302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.088393 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.088460 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:45.088871 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:45.088877 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:49:45.088945 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:49:45.089032 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":168,"threads_started":1}
I20260811 18:49:45.089232 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.084261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:49475 (local address 127.27.115.193:40573)
0811 18:49:45.084430 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:45.084433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:45.084769 (+ 336us) server_negotiation.cc:407] Connection header received
0811 18:49:45.084893 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:45.084896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:45.084959 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:45.085052 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:45.085377 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.086251 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.087703 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.088076 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.088199 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.088653 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:45.088767 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:45.088850 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:45.088993 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:49:45.089065 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:49:45.097821 30615 raft_consensus.cc:1261] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Refusing update from remote peer 1b6c6c24ac3e4b818cf22ddaac708d76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:45.097983 30285 raft_consensus.cc:1261] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Refusing update from remote peer 1b6c6c24ac3e4b818cf22ddaac708d76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:45.098496 30758 consensus_queue.cc:1035] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:45.098713 30705 consensus_queue.cc:1035] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:45.123977 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.120213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:39543 (local address 127.27.115.194:45417)
0811 18:49:45.120340 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:45.120345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:45.120366 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:49:45.120552 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:45.120554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:45.120602 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:45.120694 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:45.120952 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.121604 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.122751 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.122994 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.123133 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.123416 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:45.123511 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:45.123602 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:45.123743 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 18:49:45.123805 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:49:45.123977 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:45.120058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45417 (local address 127.27.115.193:39543)
0811 18:49:45.120237 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:45.120252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:45.120355 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:45.120709 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:45.120713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:45.120721 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:45.120830 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.120836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.121718 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.121724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.122573 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:45.122585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:45.123140 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:45.123143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:45.123240 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:45.123274 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:45.123609 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:45.123619 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:49:45.123703 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:49:45.123807 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 18:49:45.125391 30732 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:49:46.711864 30104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:46.712033 30104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:46.714854 30104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Sending DeleteTablet for 3 replicas of tablet d7f3803ee0264ce1820333157cd07354
I20260811 18:49:46.715603 30260 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f3803ee0264ce1820333157cd07354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:32794
I20260811 18:49:46.715648 30434 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f3803ee0264ce1820333157cd07354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34504
I20260811 18:49:46.715728 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f3803ee0264ce1820333157cd07354 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46284
I20260811 18:49:46.716035 30793 tablet_replica.cc:330] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: stopping tablet replica
I20260811 18:49:46.716221 30793 raft_consensus.cc:2229] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:49:46.716390 30795 tablet_replica.cc:330] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: stopping tablet replica
I20260811 18:49:46.716485 30794 tablet_replica.cc:330] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: stopping tablet replica
I20260811 18:49:46.716648 30794 raft_consensus.cc:2229] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:46.716650 30795 raft_consensus.cc:2229] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:46.716769 30793 raft_consensus.cc:2258] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.716811 30794 raft_consensus.cc:2258] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.716913 30795 raft_consensus.cc:2258] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.717610 30794 ts_tablet_manager.cc:1905] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.717633 30793 ts_tablet_manager.cc:1905] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.721335 30794 ts_tablet_manager.cc:1918] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:49:46.721442 30794 log.cc:1196] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/d7f3803ee0264ce1820333157cd07354
I20260811 18:49:46.721822 30794 ts_tablet_manager.cc:1939] T d7f3803ee0264ce1820333157cd07354 P 8c09826cd05345aaa05ab46e8029480f: Deleting consensus metadata
I20260811 18:49:46.721999 30793 ts_tablet_manager.cc:1918] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:49:46.722209 30793 log.cc:1196] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/d7f3803ee0264ce1820333157cd07354
I20260811 18:49:46.722800 30090 catalog_manager.cc:4841] TS 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877): tablet d7f3803ee0264ce1820333157cd07354 (table test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]) successfully deleted
I20260811 18:49:46.723137 30793 ts_tablet_manager.cc:1939] T d7f3803ee0264ce1820333157cd07354 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting consensus metadata
I20260811 18:49:46.723183 30795 ts_tablet_manager.cc:1905] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.723969 30089 catalog_manager.cc:4841] TS 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417): tablet d7f3803ee0264ce1820333157cd07354 (table test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]) successfully deleted
I20260811 18:49:46.725670 30795 ts_tablet_manager.cc:1918] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:49:46.725780 30795 log.cc:1196] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/d7f3803ee0264ce1820333157cd07354
I20260811 18:49:46.726028 30795 ts_tablet_manager.cc:1939] T d7f3803ee0264ce1820333157cd07354 P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting consensus metadata
I20260811 18:49:46.726835 30089 catalog_manager.cc:4841] TS 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573): tablet d7f3803ee0264ce1820333157cd07354 (table test-table [id=80ef1e2f3e0946d0be31f17d692cd83e]) successfully deleted
Aug 11 18:49:46 dist-test-slave-9jjr krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474186, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:49:46 dist-test-slave-9jjr krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474186, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:46.759836 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:46.739786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.27.115.254:42255)
0811 18:49:46.740178 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:46.740186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:46.740206 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:49:46.740334 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:46.740336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:46.740397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:46.740483 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:46.741456 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:46.742111 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:46.743319 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:46.743616 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:46.743731 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:46.754083 (+ 10352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:46.754088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:46.754091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:46.754251 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:46.756730 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:46.757622 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:46.757627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:46.757628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:46.757798 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:46.758501 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:46.758504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:46.758506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:46.758787 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:46.758937 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:46.759486 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 18:49:46.759669 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":133,"threads_started":1}
I20260811 18:49:46.764030 30104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:46.764663 30104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:46.770783 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 7b22d85a98ab45bc97a68212105b54fb (DEFAULT_TABLE table=test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.770869 30434 tablet_service.cc:1467] Processing CreateTablet for tablet 7b22d85a98ab45bc97a68212105b54fb (DEFAULT_TABLE table=test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.770855 30260 tablet_service.cc:1467] Processing CreateTablet for tablet 7b22d85a98ab45bc97a68212105b54fb (DEFAULT_TABLE table=test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.771196 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b22d85a98ab45bc97a68212105b54fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.771193 30434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b22d85a98ab45bc97a68212105b54fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.771379 30260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b22d85a98ab45bc97a68212105b54fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.774426 30807 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Bootstrap starting.
I20260811 18:49:46.774418 30805 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Bootstrap starting.
I20260811 18:49:46.774418 30806 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Bootstrap starting.
I20260811 18:49:46.775415 30807 tablet_bootstrap.cc:654] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.777508 30806 tablet_bootstrap.cc:654] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.777590 30805 tablet_bootstrap.cc:654] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.778844 30807 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: No bootstrap required, opened a new log
I20260811 18:49:46.779146 30807 ts_tablet_manager.cc:1397] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:49:46.779848 30807 raft_consensus.cc:348] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.780028 30807 raft_consensus.cc:374] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.780071 30807 raft_consensus.cc:729] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.780237 30807 consensus_queue.cc:260] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.780416 30806 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: No bootstrap required, opened a new log
I20260811 18:49:46.780568 30805 tablet_bootstrap.cc:492] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: No bootstrap required, opened a new log
I20260811 18:49:46.780611 30806 ts_tablet_manager.cc:1397] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:49:46.780712 30805 ts_tablet_manager.cc:1397] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 18:49:46.781199 30807 ts_tablet_manager.cc:1428] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:49:46.781307 30805 raft_consensus.cc:348] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.781438 30805 raft_consensus.cc:374] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.781472 30805 raft_consensus.cc:729] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.781399 30806 raft_consensus.cc:348] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.781538 30806 raft_consensus.cc:374] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.781575 30806 raft_consensus.cc:729] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.781615 30805 consensus_queue.cc:260] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.781703 30806 consensus_queue.cc:260] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.781932 30805 ts_tablet_manager.cc:1428] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:46.782300 30806 ts_tablet_manager.cc:1428] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:49:46.806974 30777 raft_consensus.cc:482] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:46.807197 30777 raft_consensus.cc:504] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.807721 30777 leader_election.cc:288] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:46.808180 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b22d85a98ab45bc97a68212105b54fb" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f" is_pre_election: true
I20260811 18:49:46.808364 30615 raft_consensus.cc:2454] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:46.808698 30209 leader_election.cc:302] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cdf73b73304a3ea4034ecf2e4e1201, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:46.808624 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b22d85a98ab45bc97a68212105b54fb" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" is_pre_election: true
I20260811 18:49:46.808902 30454 raft_consensus.cc:2454] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 0.
I20260811 18:49:46.808904 30777 raft_consensus.cc:2781] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:46.809072 30777 raft_consensus.cc:482] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:46.809131 30777 raft_consensus.cc:3037] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:46.810267 30777 raft_consensus.cc:504] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.810683 30777 leader_election.cc:288] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Requested vote from peers 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417), 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877)
I20260811 18:49:46.811139 30615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b22d85a98ab45bc97a68212105b54fb" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c09826cd05345aaa05ab46e8029480f"
I20260811 18:49:46.811209 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b22d85a98ab45bc97a68212105b54fb" candidate_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
I20260811 18:49:46.811404 30454 raft_consensus.cc:3037] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:46.811416 30615 raft_consensus.cc:3037] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:46.812532 30615 raft_consensus.cc:2454] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:46.812537 30454 raft_consensus.cc:2454] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cdf73b73304a3ea4034ecf2e4e1201 in term 1.
I20260811 18:49:46.812870 30208 leader_election.cc:302] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 29cdf73b73304a3ea4034ecf2e4e1201; no voters:
I20260811 18:49:46.813181 30777 raft_consensus.cc:2781] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:46.813304 30777 raft_consensus.cc:686] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 LEADER]: Becoming Leader. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Running, Role: LEADER
I20260811 18:49:46.813494 30777 consensus_queue.cc:237] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.816212 30104 catalog_manager.cc:5495] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193). New cstate: current_term: 1 leader_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } health_report { overall_health: UNKNOWN } } }
I20260811 18:49:46.844480 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:46.839581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.27.115.193:40573)
0811 18:49:46.839941 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:46.839951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:46.839966 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:49:46.840012 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:46.840014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:46.840060 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:46.840141 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:46.840588 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:46.841282 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:46.842803 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:46.843109 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:46.843240 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:46.843711 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:46.843849 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:46.843954 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:46.844217 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 18:49:46.844316 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":154,"threads_started":1}
I20260811 18:49:46.855343 30454 raft_consensus.cc:1261] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:46.855931 30615 raft_consensus.cc:1261] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Refusing update from remote peer 29cdf73b73304a3ea4034ecf2e4e1201: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:46.856135 30710 consensus_queue.cc:1035] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:46.856505 30772 consensus_queue.cc:1035] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:46.895195 30104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:46.895362 30104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:46.897806 30104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Sending DeleteTablet for 3 replicas of tablet 7b22d85a98ab45bc97a68212105b54fb
I20260811 18:49:46.898618 30260 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b22d85a98ab45bc97a68212105b54fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:32794
I20260811 18:49:46.898701 30434 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b22d85a98ab45bc97a68212105b54fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34504
I20260811 18:49:46.898699 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b22d85a98ab45bc97a68212105b54fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:46 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46284
I20260811 18:49:46.898862 30795 tablet_replica.cc:330] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: stopping tablet replica
I20260811 18:49:46.898957 30793 tablet_replica.cc:330] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: stopping tablet replica
I20260811 18:49:46.899011 30795 raft_consensus.cc:2229] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:49:46.899097 30793 raft_consensus.cc:2229] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:46.899129 30794 tablet_replica.cc:330] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: stopping tablet replica
I20260811 18:49:46.899241 30794 raft_consensus.cc:2229] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:46.899250 30793 raft_consensus.cc:2258] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.899261 30795 raft_consensus.cc:2258] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.899363 30794 raft_consensus.cc:2258] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:46.899549 30104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:46.900116 30104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:46.900216 30795 ts_tablet_manager.cc:1905] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.900238 30794 ts_tablet_manager.cc:1905] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.901497 30793 ts_tablet_manager.cc:1905] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:46.903273 30795 ts_tablet_manager.cc:1918] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:46.903286 30794 ts_tablet_manager.cc:1918] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:46.903405 30795 log.cc:1196] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/7b22d85a98ab45bc97a68212105b54fb
I20260811 18:49:46.903403 30794 log.cc:1196] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/7b22d85a98ab45bc97a68212105b54fb
I20260811 18:49:46.903703 30794 ts_tablet_manager.cc:1939] T 7b22d85a98ab45bc97a68212105b54fb P 8c09826cd05345aaa05ab46e8029480f: Deleting consensus metadata
I20260811 18:49:46.903713 30795 ts_tablet_manager.cc:1939] T 7b22d85a98ab45bc97a68212105b54fb P 29cdf73b73304a3ea4034ecf2e4e1201: Deleting consensus metadata
I20260811 18:49:46.904635 30090 catalog_manager.cc:4841] TS 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877): tablet 7b22d85a98ab45bc97a68212105b54fb (table test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]) successfully deleted
I20260811 18:49:46.904639 30089 catalog_manager.cc:4841] TS 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573): tablet 7b22d85a98ab45bc97a68212105b54fb (table test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]) successfully deleted
I20260811 18:49:46.905606 30793 ts_tablet_manager.cc:1918] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:49:46.905750 30793 log.cc:1196] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/7b22d85a98ab45bc97a68212105b54fb
I20260811 18:49:46.906075 30793 ts_tablet_manager.cc:1939] T 7b22d85a98ab45bc97a68212105b54fb P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting consensus metadata
I20260811 18:49:46.906905 30089 catalog_manager.cc:4841] TS 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417): tablet 7b22d85a98ab45bc97a68212105b54fb (table test-table [id=1c578e4173cc41a3b51ed7ee0add78ba]) successfully deleted
I20260811 18:49:46.908123 30260 tablet_service.cc:1467] Processing CreateTablet for tablet b1f989f63a154f76b780b223f2233488 (DEFAULT_TABLE table=test-table [id=018f0013996d469285a0fe2a0cf99f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.908123 30434 tablet_service.cc:1467] Processing CreateTablet for tablet b1f989f63a154f76b780b223f2233488 (DEFAULT_TABLE table=test-table [id=018f0013996d469285a0fe2a0cf99f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.908356 30593 tablet_service.cc:1467] Processing CreateTablet for tablet b1f989f63a154f76b780b223f2233488 (DEFAULT_TABLE table=test-table [id=018f0013996d469285a0fe2a0cf99f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:46.908439 30260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f989f63a154f76b780b223f2233488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.908439 30434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f989f63a154f76b780b223f2233488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.908696 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f989f63a154f76b780b223f2233488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:46.911876 30807 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Bootstrap starting.
I20260811 18:49:46.911876 30805 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: Bootstrap starting.
I20260811 18:49:46.912828 30805 tablet_bootstrap.cc:654] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.912828 30807 tablet_bootstrap.cc:654] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.913815 30806 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Bootstrap starting.
I20260811 18:49:46.914801 30805 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: No bootstrap required, opened a new log
I20260811 18:49:46.914917 30805 ts_tablet_manager.cc:1397] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:49:46.914955 30806 tablet_bootstrap.cc:654] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:46.915462 30805 raft_consensus.cc:348] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.915586 30805 raft_consensus.cc:374] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.915620 30805 raft_consensus.cc:729] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cdf73b73304a3ea4034ecf2e4e1201, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.915788 30805 consensus_queue.cc:260] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.916087 30805 ts_tablet_manager.cc:1428] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:49:46.916260 30807 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: No bootstrap required, opened a new log
I20260811 18:49:46.916358 30807 ts_tablet_manager.cc:1397] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:49:46.916854 30807 raft_consensus.cc:348] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.916992 30807 raft_consensus.cc:374] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.917029 30807 raft_consensus.cc:729] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6c6c24ac3e4b818cf22ddaac708d76, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.917177 30807 consensus_queue.cc:260] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.917804 30806 tablet_bootstrap.cc:492] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: No bootstrap required, opened a new log
I20260811 18:49:46.917919 30806 ts_tablet_manager.cc:1397] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:49:46.918552 30806 raft_consensus.cc:348] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.918669 30806 raft_consensus.cc:374] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:46.918700 30806 raft_consensus.cc:729] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Initialized, Role: FOLLOWER
I20260811 18:49:46.918862 30806 consensus_queue.cc:260] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:46.919183 30806 ts_tablet_manager.cc:1428] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:46.919528 30807 ts_tablet_manager.cc:1428] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:49:47.042874 30811 raft_consensus.cc:482] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:47.043054 30811 raft_consensus.cc:504] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:47.044457 30811 leader_election.cc:288] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573), 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417)
I20260811 18:49:47.049151 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.045147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:34205 (local address 127.27.115.193:40573)
0811 18:49:47.045309 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:47.045316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:47.045336 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:49:47.045563 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:47.045567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:47.045624 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:47.045669 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:47.046043 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.046706 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048033 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.048519 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048667 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.048734 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:47.048932 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 18:49:47.049011 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:49:47.049201 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.044820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.27.115.195:34205)
0811 18:49:47.045144 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:47.045179 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:47.045375 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:47.045807 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:47.045809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:47.045825 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:47.045926 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.045930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.046827 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.046833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.047901 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.047911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.048678 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.048797 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.048816 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:49:47.048911 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:49:47.048999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260811 18:49:47.049345 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.045360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:43031 (local address 127.27.115.194:45417)
0811 18:49:47.045663 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:47.045678 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:47.045693 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:49:47.045794 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:47.045798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:47.045850 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:47.045945 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:47.046379 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.046982 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048191 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.048638 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048785 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.048840 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:47.049090 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 18:49:47.049180 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":109,"threads_started":1}
I20260811 18:49:47.049401 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.045220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45417 (local address 127.27.115.195:43031)
0811 18:49:47.045546 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:47.045562 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:47.045690 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:47.045960 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:47.045963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:47.045982 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:47.046255 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.046264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.047105 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.047109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.048050 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.048060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.048771 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.048775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.048949 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.048970 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:49:47.049067 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:49:47.049145 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":24,"thread_start_us":112,"threads_started":1}
I20260811 18:49:47.049718 30285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f989f63a154f76b780b223f2233488" candidate_uuid: "8c09826cd05345aaa05ab46e8029480f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" is_pre_election: true
I20260811 18:49:47.049892 30285 raft_consensus.cc:2454] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c09826cd05345aaa05ab46e8029480f in term 0.
I20260811 18:49:47.049891 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f989f63a154f76b780b223f2233488" candidate_uuid: "8c09826cd05345aaa05ab46e8029480f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" is_pre_election: true
I20260811 18:49:47.050019 30454 raft_consensus.cc:2454] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c09826cd05345aaa05ab46e8029480f in term 0.
I20260811 18:49:47.050392 30540 leader_election.cc:302] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cdf73b73304a3ea4034ecf2e4e1201, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:47.050714 30811 raft_consensus.cc:2781] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:47.050773 30811 raft_consensus.cc:482] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:47.050808 30811 raft_consensus.cc:3037] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:47.052193 30811 raft_consensus.cc:504] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:47.052626 30811 leader_election.cc:288] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [CANDIDATE]: Term 1 election: Requested vote from peers 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573), 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417)
I20260811 18:49:47.053210 30454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f989f63a154f76b780b223f2233488" candidate_uuid: "8c09826cd05345aaa05ab46e8029480f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76"
I20260811 18:49:47.053226 30285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f989f63a154f76b780b223f2233488" candidate_uuid: "8c09826cd05345aaa05ab46e8029480f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cdf73b73304a3ea4034ecf2e4e1201"
I20260811 18:49:47.053339 30454 raft_consensus.cc:3037] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:47.053398 30285 raft_consensus.cc:3037] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:47.054661 30454 raft_consensus.cc:2454] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c09826cd05345aaa05ab46e8029480f in term 1.
I20260811 18:49:47.054715 30285 raft_consensus.cc:2454] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c09826cd05345aaa05ab46e8029480f in term 1.
I20260811 18:49:47.055131 30540 leader_election.cc:302] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6c6c24ac3e4b818cf22ddaac708d76, 8c09826cd05345aaa05ab46e8029480f; no voters:
I20260811 18:49:47.055460 30811 raft_consensus.cc:2781] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:47.055768 30811 raft_consensus.cc:686] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 1 LEADER]: Becoming Leader. State: Replica: 8c09826cd05345aaa05ab46e8029480f, State: Running, Role: LEADER
I20260811 18:49:47.056097 30811 consensus_queue.cc:237] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } }
I20260811 18:49:47.059137 30106 catalog_manager.cc:5495] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c09826cd05345aaa05ab46e8029480f (127.27.115.195). New cstate: current_term: 1 leader_uuid: "8c09826cd05345aaa05ab46e8029480f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c09826cd05345aaa05ab46e8029480f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 37877 } health_report { overall_health: HEALTHY } } }
I20260811 18:49:47.082276 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.077197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.27.115.195:37877)
0811 18:49:47.077579 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:47.077586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:47.077598 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:49:47.077708 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:47.077710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:47.077770 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:47.077979 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:47.078322 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.079189 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.080080 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.080469 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.080737 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.080984 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:47.081516 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:47.081682 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:47.081887 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 18:49:47.081969 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":186,"threads_started":1}
I20260811 18:49:47.096503 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.091961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:45121 (local address 127.27.115.193:40573)
0811 18:49:47.092140 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:47.092147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:47.092418 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:49:47.092521 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:47.092523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:47.092578 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:47.092661 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:47.093017 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.093552 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.095258 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.095505 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.095599 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.096031 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:47.096115 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:47.096174 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:47.096307 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 18:49:47.096366 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:49:47.096782 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.091779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.27.115.195:45121)
0811 18:49:47.092288 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:47.092303 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:47.092419 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:47.092761 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:47.092764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:47.092773 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:47.092898 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.092903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.093834 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.093837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.095088 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.095101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.095668 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.095671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.095819 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.095881 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:47.096208 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:47.096215 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:49:47.096275 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:49:47.096377 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":211,"threads_started":1}
I20260811 18:49:47.104831 30285 raft_consensus.cc:1261] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Refusing update from remote peer 8c09826cd05345aaa05ab46e8029480f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:47.105082 30454 raft_consensus.cc:1261] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Refusing update from remote peer 8c09826cd05345aaa05ab46e8029480f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:47.105702 30829 consensus_queue.cc:1035] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [LEADER]: Connected to new peer: Peer: permanent_uuid: "29cdf73b73304a3ea4034ecf2e4e1201" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:47.106009 30811 consensus_queue.cc:1035] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6c6c24ac3e4b818cf22ddaac708d76" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:47.128796 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.124146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:55707 (local address 127.27.115.195:37877)
0811 18:49:47.124290 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:47.124295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:47.124799 (+ 504us) server_negotiation.cc:407] Connection header received
0811 18:49:47.124900 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:47.124903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:47.124958 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:47.125193 (+ 235us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:47.125407 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.125968 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.127370 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.127683 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.127859 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.128227 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:47.128333 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:47.128420 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:47.128557 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:49:47.128633 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:49:47.128798 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:47.124040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:37877 (local address 127.27.115.193:55707)
0811 18:49:47.124621 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:47.124637 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:47.124808 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:47.125168 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:47.125171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:47.125179 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:49:47.125297 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.125302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.126283 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.126287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.127214 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:47.127228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:47.127918 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:47.127923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:47.128016 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:47.128063 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:49:47.128427 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:49:47.128437 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:49:47.128530 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:49:47.128635 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":223,"threads_started":1}
W20260811 18:49:47.129906 30773 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:49:48.718695 30106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:48.718868 30106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:48.721761 30106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4b6ff8b6984410a8251165e386c12bf: Sending DeleteTablet for 3 replicas of tablet b1f989f63a154f76b780b223f2233488
I20260811 18:49:48.722471 30260 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f989f63a154f76b780b223f2233488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:48 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:32794
I20260811 18:49:48.722572 30434 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f989f63a154f76b780b223f2233488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:48 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34504
I20260811 18:49:48.722609 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f989f63a154f76b780b223f2233488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:48 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46284
I20260811 18:49:48.722965 30874 tablet_replica.cc:330] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: stopping tablet replica
I20260811 18:49:48.722993 30872 tablet_replica.cc:330] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201: stopping tablet replica
I20260811 18:49:48.723156 30874 raft_consensus.cc:2229] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:48.723186 30872 raft_consensus.cc:2229] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:48.723304 30874 raft_consensus.cc:2258] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:48.723332 30872 raft_consensus.cc:2258] T b1f989f63a154f76b780b223f2233488 P 29cdf73b73304a3ea4034ecf2e4e1201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:48.723378 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 30189
I20260811 18:49:48.725282 30873 tablet_replica.cc:330] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: stopping tablet replica
I20260811 18:49:48.725412 30873 raft_consensus.cc:2229] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:49:48.725744 30873 raft_consensus.cc:2258] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:48.726897 30873 ts_tablet_manager.cc:1905] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:48.731102 30873 ts_tablet_manager.cc:1918] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:49:48.731246 30873 log.cc:1196] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/b1f989f63a154f76b780b223f2233488
I20260811 18:49:48.731323 30874 ts_tablet_manager.cc:1905] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:48.731657 30873 ts_tablet_manager.cc:1939] T b1f989f63a154f76b780b223f2233488 P 8c09826cd05345aaa05ab46e8029480f: Deleting consensus metadata
I20260811 18:49:48.732964 30090 catalog_manager.cc:4841] TS 8c09826cd05345aaa05ab46e8029480f (127.27.115.195:37877): tablet b1f989f63a154f76b780b223f2233488 (table test-table [id=018f0013996d469285a0fe2a0cf99f57]) successfully deleted
I20260811 18:49:48.733963 30874 ts_tablet_manager.cc:1918] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:49:48.734081 30874 log.cc:1196] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/b1f989f63a154f76b780b223f2233488
I20260811 18:49:48.734426 30874 ts_tablet_manager.cc:1939] T b1f989f63a154f76b780b223f2233488 P 1b6c6c24ac3e4b818cf22ddaac708d76: Deleting consensus metadata
I20260811 18:49:48.735271 30089 catalog_manager.cc:4841] TS 1b6c6c24ac3e4b818cf22ddaac708d76 (127.27.115.194:45417): tablet b1f989f63a154f76b780b223f2233488 (table test-table [id=018f0013996d469285a0fe2a0cf99f57]) successfully deleted
W20260811 18:49:48.738421 30089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:40573 (error 108)
I20260811 18:49:48.738557 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 30353
I20260811 18:49:48.741899 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:48.741336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40573 (local address 127.0.0.1:50192)
0811 18:49:48.741702 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:48.741792 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:40573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":131,"threads_started":1}
W20260811 18:49:48.742312 30089 catalog_manager.cc:4584] TS 29cdf73b73304a3ea4034ecf2e4e1201 (127.27.115.193:40573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1f989f63a154f76b780b223f2233488: Network error: Client connection negotiation failed: client connection to 127.27.115.193:40573: connect: Connection refused (error 111)
I20260811 18:49:48.750844 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 30520
I20260811 18:49:48.762581 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 30071
2026-08-11T18:49:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6677 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:48 dist-test-slave-9jjr krb5kdc[30881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:48 dist-test-slave-9jjr krb5kdc[30881](info): set up 2 sockets
Aug 11 18:49:48 dist-test-slave-9jjr krb5kdc[30881](info): commencing operation
krb5kdc: starting...
W20260811 18:49:49.999207 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 18:49:50.220176 28111 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.221s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 18:49:50.445354 28111 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.144s user 0.000s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/test-user.keytab.
W20260811 18:49:50.909618 28111 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.464s user 0.000s sys 0.002s
Aug 11 18:49:50 dist-test-slave-9jjr krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
W20260811 18:49:51.522924 28111 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.27.115.254: real 0.583s user 0.000s sys 0.005s
I20260811 18:49:51.523533 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33457
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:33457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:51.650995 30895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:51.651345 30895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:51.651393 30895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:51.655843 30895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:51.655939 30895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:51.655959 30895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:51.655977 30895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:51.655993 30895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:51.662292 30895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:33457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:51.663784 30895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:51.664954 30895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:49:51.674798 30895 server_base.cc:1034] running on GCE node
W20260811 18:49:51.674737 30904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:51.674784 30902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:51.674737 30901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:51.675757 30895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:51.676987 30895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:51.678201 30895 hybrid_clock.cc:648] HybridClock initialized: now 1786474191678156 us; error 60 us; skew 500 ppm
Aug 11 18:49:51 dist-test-slave-9jjr krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474191, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:51.681314 30895 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:51.682500 30895 webserver.cc:460] Webserver started at http://127.27.115.254:32777/ using document root <none> and password file <none>
I20260811 18:49:51.683185 30895 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:51.683290 30895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:51.683553 30895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:51.685966 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "c256b697537d4fcfacf19b6186f065a8"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "f82dbd0d6d782cc23b4d555cc3fd8941"
server_key_iv: "a1e27890fb8250c12e9abdd18e406125"
server_key_version: "encryptionkey@0"
I20260811 18:49:51.686740 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "c256b697537d4fcfacf19b6186f065a8"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "f82dbd0d6d782cc23b4d555cc3fd8941"
server_key_iv: "a1e27890fb8250c12e9abdd18e406125"
server_key_version: "encryptionkey@0"
I20260811 18:49:51.691406 30895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:49:51.694424 30911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:51.695870 30895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:49:51.696059 30895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "c256b697537d4fcfacf19b6186f065a8"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "f82dbd0d6d782cc23b4d555cc3fd8941"
server_key_iv: "a1e27890fb8250c12e9abdd18e406125"
server_key_version: "encryptionkey@0"
I20260811 18:49:51.696200 30895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:51.712759 30895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:51.716840 30895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:51.717099 30895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:51.725662 30895 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:33457
I20260811 18:49:51.725679 30969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:33457 every 8 connection(s)
I20260811 18:49:51.726931 30895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:51.730589 30971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:51.732788 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 30895
I20260811 18:49:51.732923 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:51.733224 28111 external_mini_cluster.cc:1428] Setting key d2079727475206e811677f76e9d7a36b
I20260811 18:49:51.738696 30971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8: Bootstrap starting.
I20260811 18:49:51.741817 30971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:51.742837 30971 log.cc:824] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:49:51 dist-test-slave-9jjr krb5kdc[30881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:51.745783 30971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8: No bootstrap required, opened a new log
I20260811 18:49:51.751364 30971 raft_consensus.cc:348] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } }
I20260811 18:49:51.751796 30971 raft_consensus.cc:374] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:51.751888 30971 raft_consensus.cc:729] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c256b697537d4fcfacf19b6186f065a8, State: Initialized, Role: FOLLOWER
I20260811 18:49:51.752636 30971 consensus_queue.cc:260] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } }
I20260811 18:49:51.752861 30971 raft_consensus.cc:388] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:51.752929 30971 raft_consensus.cc:482] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:51.752997 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:51.734933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.27.115.254:33457)
0811 18:49:51.735449 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:51.735471 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:51.735519 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:49:51.736450 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:51.736478 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:51.736958 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:51.737473 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:51.738309 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:51.739820 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:51.740824 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:51.741544 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:51.741750 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:51.745081 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:51.745089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:51.745108 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:51.745143 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:51.748853 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:51.749532 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:51.749540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:51.749547 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:51.749724 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:51.750019 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:51.750022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:51.750024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:51.750736 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:51.750983 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:51.751321 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 18:49:51.751693 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":138,"threads_started":1}
I20260811 18:49:51.753041 30971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:51.754274 30971 raft_consensus.cc:504] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } }
I20260811 18:49:51.754678 30971 leader_election.cc:302] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c256b697537d4fcfacf19b6186f065a8; no voters:
I20260811 18:49:51.755081 30971 leader_election.cc:288] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:51.755192 30979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:51.755415 30979 raft_consensus.cc:686] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [term 1 LEADER]: Becoming Leader. State: Replica: c256b697537d4fcfacf19b6186f065a8, State: Running, Role: LEADER
I20260811 18:49:51.755869 30979 consensus_queue.cc:237] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } }
I20260811 18:49:51.756675 30971 sys_catalog.cc:564] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:51.758513 30981 sys_catalog.cc:455] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c256b697537d4fcfacf19b6186f065a8. Latest consensus state: current_term: 1 leader_uuid: "c256b697537d4fcfacf19b6186f065a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } } }
I20260811 18:49:51.758708 30981 sys_catalog.cc:458] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:51.758490 30980 sys_catalog.cc:455] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c256b697537d4fcfacf19b6186f065a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256b697537d4fcfacf19b6186f065a8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33457 } } }
I20260811 18:49:51.758791 30980 sys_catalog.cc:458] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:51.759282 30993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:51.763629 30993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:51.770190 30993 catalog_manager.cc:1347] Generated new cluster ID: 1605b50d90b24902ae73868f87160123
I20260811 18:49:51.770323 30993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:51.777204 30993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:51.778643 30993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:51.789565 30993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c256b697537d4fcfacf19b6186f065a8: Generated new TSK 0
I20260811 18:49:51.790788 30993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:51.826915 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:51.960194 31003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:51.960506 31003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:51.960553 31003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:51.964903 31003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:51.964983 31003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:51.965046 31003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:51.970793 31003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:51.971995 31003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:51.973160 31003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:51.984164 31011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:51.984138 31008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:51.984143 31014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:51.984539 31003 server_base.cc:1034] running on GCE node
I20260811 18:49:51.985000 31003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:51.985747 31003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:51.986941 31003 hybrid_clock.cc:648] HybridClock initialized: now 1786474191986906 us; error 49 us; skew 500 ppm
Aug 11 18:49:51 dist-test-slave-9jjr krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474191, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:51.994283 31003 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:51.995357 31003 webserver.cc:460] Webserver started at http://127.27.115.193:33655/ using document root <none> and password file <none>
I20260811 18:49:51.996001 31003 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:51.996102 31003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:51.996356 31003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:51.998657 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "d8d214f223fd4786a0a21707e87cdcf4"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "0c072c9a126a1fec56530fc6527257bb"
server_key_iv: "78bba53b87225d9df323d34c4cf9cf15"
server_key_version: "encryptionkey@0"
I20260811 18:49:51.999253 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "d8d214f223fd4786a0a21707e87cdcf4"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "0c072c9a126a1fec56530fc6527257bb"
server_key_iv: "78bba53b87225d9df323d34c4cf9cf15"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.003831 31003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:49:52.006703 31025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.008040 31003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:52.008212 31003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "d8d214f223fd4786a0a21707e87cdcf4"
format_stamp: "Formatted at 2026-08-11 18:49:51 on dist-test-slave-9jjr"
server_key: "0c072c9a126a1fec56530fc6527257bb"
server_key_iv: "78bba53b87225d9df323d34c4cf9cf15"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.008356 31003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:52.024641 31003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:52.028332 31003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:52.028519 31003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:52.029125 31003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:52.030056 31003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:52.030110 31003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.030236 31003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:52.030270 31003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.041159 31003 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:43225
I20260811 18:49:52.041183 31147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:43225 every 8 connection(s)
I20260811 18:49:52.042310 31003 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:52.046217 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31003
I20260811 18:49:52.046361 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:52.046726 28111 external_mini_cluster.cc:1428] Setting key 262d06b0384035c67c7925ec78587d91
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[30881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474191, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:49:52.060833 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.044698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:49683 (local address 127.27.115.254:33457)
0811 18:49:52.044937 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:52.044941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:52.045957 (+ 1016us) server_negotiation.cc:407] Connection header received
0811 18:49:52.047786 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:52.047790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:52.047841 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:52.047937 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:52.049807 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.050445 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.052038 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.052456 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.052557 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.056095 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:52.056098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:52.056100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:52.056131 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:52.057995 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.058722 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.058726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.058728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.058828 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.059304 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.059307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.059309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.059532 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:52.059659 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:52.060480 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 18:49:52.060647 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:52.062194 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.045108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33457 (local address 127.27.115.193:49683)
0811 18:49:52.045697 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:52.045747 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:52.047508 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:52.048130 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:52.048138 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:52.048754 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:52.049570 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.049597 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.050597 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.050604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.051780 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.051792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.052684 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.052692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.052998 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.053790 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:52.053819 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:52.055847 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:52.058164 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.058171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.058192 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.058564 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.058939 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.058942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.058944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.059189 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.059686 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:52.059695 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:52.060143 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 18:49:52.060414 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 18:49:52.060831 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":165,"threads_started":1}
I20260811 18:49:52.063896 31148 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33457
I20260811 18:49:52.064241 31148 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:52.064971 31148 heartbeater.cc:511] Master 127.27.115.254:33457 requested a full tablet report, sending...
I20260811 18:49:52.066982 30931 ts_manager.cc:194] Registered new tserver with Master: d8d214f223fd4786a0a21707e87cdcf4 (127.27.115.193:43225)
I20260811 18:49:52.068264 30931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:49683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:52.082177 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:52.212615 31155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:52.213049 31155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:52.213116 31155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:52.217391 31155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:52.217478 31155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:52.217541 31155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:52.223800 31155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:52.225183 31155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:52.226475 31155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:52.241788 31164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:52.241788 31163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:52.241990 31166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:52.242316 31155 server_base.cc:1034] running on GCE node
I20260811 18:49:52.242867 31155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:52.243654 31155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:52.244915 31155 hybrid_clock.cc:648] HybridClock initialized: now 1786474192244888 us; error 41 us; skew 500 ppm
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474192, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:52.248768 31155 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:52.249991 31155 webserver.cc:460] Webserver started at http://127.27.115.194:46061/ using document root <none> and password file <none>
I20260811 18:49:52.250759 31155 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:52.250877 31155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:52.251132 31155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:52.253646 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "52b1e58298ab4110b98f60c6f456ff3b"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "9521125e36eba3ab65af71e384eb8ff4"
server_key_iv: "87936b77ef85d83e122c8d9fa9024c3e"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.254452 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "52b1e58298ab4110b98f60c6f456ff3b"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "9521125e36eba3ab65af71e384eb8ff4"
server_key_iv: "87936b77ef85d83e122c8d9fa9024c3e"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.259755 31155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:49:52.262926 31173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.264595 31155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:49:52.264827 31155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "52b1e58298ab4110b98f60c6f456ff3b"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "9521125e36eba3ab65af71e384eb8ff4"
server_key_iv: "87936b77ef85d83e122c8d9fa9024c3e"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.264988 31155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:52.279523 31155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:52.283923 31155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:52.284220 31155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:52.284982 31155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:52.286056 31155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:52.286139 31155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.286275 31155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:52.286322 31155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.297909 31155 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:38073
I20260811 18:49:52.297940 31303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:38073 every 8 connection(s)
I20260811 18:49:52.299109 31155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:52.301820 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31155
I20260811 18:49:52.301925 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:52.302318 28111 external_mini_cluster.cc:1428] Setting key bf0b38741cc189814f855bc9aec1a5de
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[30881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474192, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:49:52.318470 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.301541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:38227 (local address 127.27.115.254:33457)
0811 18:49:52.302330 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:52.302337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:52.302977 (+ 640us) server_negotiation.cc:407] Connection header received
0811 18:49:52.304608 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:52.304610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:52.304688 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:52.304764 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:52.306822 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.307724 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.309126 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.309589 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.309751 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.313495 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:52.313504 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:52.313509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:52.313555 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:52.315508 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.316252 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.316256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.316258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.316356 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.317046 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.317050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.317051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.317301 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:52.317407 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:52.318023 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 18:49:52.318251 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:49:52.319558 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.302070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33457 (local address 127.27.115.194:38227)
0811 18:49:52.302791 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:52.302847 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:52.304323 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:52.304988 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:52.305002 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:52.305654 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:52.306594 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.306618 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.307937 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.307940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.308953 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.308965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.309839 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.309843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.310030 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.311047 (+ 1017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:52.311066 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:52.313125 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:52.315744 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.315750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.315767 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.316091 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.316700 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.316703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.316704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.316889 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.317416 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:52.317423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:52.317727 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 18:49:52.317914 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 18:49:52.318346 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":196,"threads_started":1}
I20260811 18:49:52.321159 31304 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33457
I20260811 18:49:52.321485 31304 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:52.322181 31304 heartbeater.cc:511] Master 127.27.115.254:33457 requested a full tablet report, sending...
I20260811 18:49:52.323486 30931 ts_manager.cc:194] Registered new tserver with Master: 52b1e58298ab4110b98f60c6f456ff3b (127.27.115.194:38073)
I20260811 18:49:52.324172 30931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:38227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:52.338852 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:52.464318 31310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:52.464622 31310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:52.464670 31310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:52.468777 31310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:52.468843 31310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:52.468904 31310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:52.474792 31310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:52.476004 31310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:52.477075 31310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:52.487394 31316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:52.487396 31315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:52.487412 31318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:52.487746 31310 server_base.cc:1034] running on GCE node
I20260811 18:49:52.488312 31310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:52.489092 31310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:52.490311 31310 hybrid_clock.cc:648] HybridClock initialized: now 1786474192490296 us; error 32 us; skew 500 ppm
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[30881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474192, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:52.493485 31310 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:52.494611 31310 webserver.cc:460] Webserver started at http://127.27.115.195:42685/ using document root <none> and password file <none>
I20260811 18:49:52.495232 31310 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:52.495332 31310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:52.495591 31310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:52.497803 31310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "cc928bdb0db64ccbbbd8ca51ec31d932"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "55490656904700ed675fb5a5aecd98bd"
server_key_iv: "bf51fe0774b5e25939d193728e2cd885"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.498452 31310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "cc928bdb0db64ccbbbd8ca51ec31d932"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "55490656904700ed675fb5a5aecd98bd"
server_key_iv: "bf51fe0774b5e25939d193728e2cd885"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.503381 31310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:49:52.506500 31326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.508042 31310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:49:52.508190 31310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "cc928bdb0db64ccbbbd8ca51ec31d932"
format_stamp: "Formatted at 2026-08-11 18:49:52 on dist-test-slave-9jjr"
server_key: "55490656904700ed675fb5a5aecd98bd"
server_key_iv: "bf51fe0774b5e25939d193728e2cd885"
server_key_version: "encryptionkey@0"
I20260811 18:49:52.508322 31310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:52.535377 31310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:52.538972 31310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:52.539152 31310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:52.539745 31310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:52.540639 31310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:52.540692 31310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.540737 31310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:52.540752 31310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:52.551949 31310 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:38523
I20260811 18:49:52.551971 31448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:38523 every 8 connection(s)
I20260811 18:49:52.553081 31310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:52.558275 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31310
I20260811 18:49:52.558435 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:52.558722 28111 external_mini_cluster.cc:1428] Setting key 7f632c7cba6d2ac74d759f8f84e7b297
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[30881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474192, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:52.570586 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.555342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:42635 (local address 127.27.115.254:33457)
0811 18:49:52.555526 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:52.555530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:52.556592 (+ 1062us) server_negotiation.cc:407] Connection header received
0811 18:49:52.558474 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:52.558478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:52.558557 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:52.558679 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:52.560510 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.561337 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.562803 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.563101 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.563237 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.566409 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:52.566412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:52.566414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:52.566444 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:52.567827 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.568474 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.568477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.568478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.568569 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:52.569131 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:52.569134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:52.569135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:52.569374 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:52.569472 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:52.570313 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 18:49:52.570453 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:49:52.571535 31451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:52.555806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33457 (local address 127.27.115.195:42635)
0811 18:49:52.556400 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:52.556455 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:52.558196 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:52.558907 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:52.558916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:52.559565 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:52.560319 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.560343 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.561522 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.561526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.562647 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:52.562660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:52.563309 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:52.563313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:52.563481 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:52.564313 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:52.564334 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:52.566178 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:52.567966 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.567972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.567988 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.568311 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.568741 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:52.568744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:52.568745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:52.568981 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:52.569520 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:52.569526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:52.569943 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 18:49:52.570215 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 18:49:52.570542 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":173,"threads_started":1}
I20260811 18:49:52.573051 31449 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33457
I20260811 18:49:52.573400 31449 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:52.574034 31449 heartbeater.cc:511] Master 127.27.115.254:33457 requested a full tablet report, sending...
I20260811 18:49:52.575124 30931 ts_manager.cc:194] Registered new tserver with Master: cc928bdb0db64ccbbbd8ca51ec31d932 (127.27.115.195:38523)
I20260811 18:49:52.575732 30931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:42635
I20260811 18:49:52.585965 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:49:52.603127 30976 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:49:52.601100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.27.115.254:33457)
0811 18:49:52.601315 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:52.601320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:52.601500 (+ 180us) server_negotiation.cc:407] Connection header received
0811 18:49:52.601710 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:52.601713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:52.601761 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:52.601850 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:52.602927 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51504: BlockingRecv error: recv got EOF from 127.0.0.1:51504 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:49:52.604198 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31003
I20260811 18:49:52.618618 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31155
I20260811 18:49:52.629771 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31310
I20260811 18:49:52.639500 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 30895
2026-08-11T18:49:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3876 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[31468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[31468](info): set up 2 sockets
Aug 11 18:49:52 dist-test-slave-9jjr krb5kdc[31468](info): commencing operation
krb5kdc: starting...
W20260811 18:49:53.776731 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:53 dist-test-slave-9jjr krb5kdc[31468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:53.926399 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43559
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:43559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:54.056629 31482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:54.056977 31482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:54.057025 31482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:54.061561 31482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:54.061668 31482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:54.061690 31482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:54.061708 31482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:54.061726 31482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:54.068115 31482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:43559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:54.069569 31482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:54.070890 31482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:54.082401 31489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:54.082401 31492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:54.082492 31487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:54.082897 31482 server_base.cc:1034] running on GCE node
I20260811 18:49:54.083741 31482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:54.085317 31482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:54.086566 31482 hybrid_clock.cc:648] HybridClock initialized: now 1786474194086530 us; error 58 us; skew 500 ppm
Aug 11 18:49:54 dist-test-slave-9jjr krb5kdc[31468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474194, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:54.090613 31482 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:54.091806 31482 webserver.cc:460] Webserver started at http://127.27.115.254:42127/ using document root <none> and password file <none>
I20260811 18:49:54.092514 31482 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:54.092636 31482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:54.092911 31482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:54.095470 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "5882edc5cddc49e492ed6e6cdc7d03b6"
format_stamp: "Formatted at 2026-08-11 18:49:54 on dist-test-slave-9jjr"
server_key: "f4919026a660500f5c09473e0b6f87a7"
server_key_iv: "e353064b6c6a4eded239771f585d262f"
server_key_version: "encryptionkey@0"
I20260811 18:49:54.096319 31482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "5882edc5cddc49e492ed6e6cdc7d03b6"
format_stamp: "Formatted at 2026-08-11 18:49:54 on dist-test-slave-9jjr"
server_key: "f4919026a660500f5c09473e0b6f87a7"
server_key_iv: "e353064b6c6a4eded239771f585d262f"
server_key_version: "encryptionkey@0"
I20260811 18:49:54.101506 31482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:49:54.105096 31499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:54.107224 31482 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:54.107534 31482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "5882edc5cddc49e492ed6e6cdc7d03b6"
format_stamp: "Formatted at 2026-08-11 18:49:54 on dist-test-slave-9jjr"
server_key: "f4919026a660500f5c09473e0b6f87a7"
server_key_iv: "e353064b6c6a4eded239771f585d262f"
server_key_version: "encryptionkey@0"
I20260811 18:49:54.107741 31482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:54.137899 31482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:54.141870 31482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:54.142174 31482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:54.150465 31482 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:43559
I20260811 18:49:54.150856 31563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:43559 every 8 connection(s)
I20260811 18:49:54.151666 31482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:54.155130 31564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:54.156798 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31482
I20260811 18:49:54.156932 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:54.157236 28111 external_mini_cluster.cc:1428] Setting key debbba0c8c4a7a2576236d142145ad8d
I20260811 18:49:54.162738 31564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6: Bootstrap starting.
I20260811 18:49:54.165876 31564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:54.167021 31564 log.cc:824] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:49:54 dist-test-slave-9jjr krb5kdc[31468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:54.169661 31564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6: No bootstrap required, opened a new log
I20260811 18:49:54.173606 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:54.158812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.27.115.254:43559)
0811 18:49:54.159281 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:54.159298 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:54.159336 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:49:54.160035 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:54.160055 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:54.160431 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:54.160748 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:54.161768 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:54.162932 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:54.164289 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:54.164799 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:54.164940 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:54.168330 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:54.168336 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:54.168351 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:54.168382 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:54.170422 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:54.170954 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:54.170961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:54.170966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:54.171088 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:54.171396 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:54.171400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:54.171402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:54.171862 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:54.172082 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:54.172417 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 18:49:54.172766 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":160,"threads_started":1}
I20260811 18:49:54.173727 31564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } }
I20260811 18:49:54.174052 31564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:54.174166 31564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5882edc5cddc49e492ed6e6cdc7d03b6, State: Initialized, Role: FOLLOWER
I20260811 18:49:54.174746 31564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } }
I20260811 18:49:54.174932 31564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:54.175001 31564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:54.175104 31564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:54.176476 31564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } }
I20260811 18:49:54.176892 31564 leader_election.cc:302] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5882edc5cddc49e492ed6e6cdc7d03b6; no voters:
I20260811 18:49:54.177089 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:49:54.177335 31564 leader_election.cc:288] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:54.177438 31569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:54.177623 31569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [term 1 LEADER]: Becoming Leader. State: Replica: 5882edc5cddc49e492ed6e6cdc7d03b6, State: Running, Role: LEADER
I20260811 18:49:54.178025 31569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } }
I20260811 18:49:54.178848 31564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:54.179382 31569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5882edc5cddc49e492ed6e6cdc7d03b6. Latest consensus state: current_term: 1 leader_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } } }
I20260811 18:49:54.179602 31569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:54.179865 31570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5882edc5cddc49e492ed6e6cdc7d03b6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43559 } } }
I20260811 18:49:54.180011 31570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:54.180214 31578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:54.184192 31578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:54.191638 31578 catalog_manager.cc:1347] Generated new cluster ID: e93fb20ef8444d6eb067c5fec0a99afd
I20260811 18:49:54.191748 31578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:54.203580 31578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:54.204998 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:54.198623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.27.115.254:43559)
0811 18:49:54.198794 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:54.198799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:54.199081 (+ 282us) server_negotiation.cc:407] Connection header received
0811 18:49:54.199222 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:54.199226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:54.199289 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:54.199406 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:54.200067 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:54.200930 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:54.202014 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:54.202380 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:54.202509 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:54.203840 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:54.203844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:54.203846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:49:54.204488 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:54.204529 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:49:54.204553 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:54.204653 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:54.204786 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 18:49:54.204855 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:49:54.205042 31578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:54.210505 31578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5882edc5cddc49e492ed6e6cdc7d03b6: Generated new TSK 0
I20260811 18:49:54.211741 31578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:49:54.221477 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31482
2026-08-11T18:49:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1577 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:54 dist-test-slave-9jjr krb5kdc[31603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:54 dist-test-slave-9jjr krb5kdc[31603](info): set up 2 sockets
Aug 11 18:49:54 dist-test-slave-9jjr krb5kdc[31603](info): commencing operation
krb5kdc: starting...
W20260811 18:49:55.362746 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:55 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:55.512620 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39333
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:55.638309 31616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:55.638664 31616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:55.638710 31616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:55.644232 31616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:55.644372 31616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:55.644415 31616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:55.644443 31616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:55.644471 31616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:55.651629 31616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:55.653017 31616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:55.654335 31616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:55.664887 31623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:55.664897 31626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:55.664929 31624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:55.665616 31616 server_base.cc:1034] running on GCE node
I20260811 18:49:55.666407 31616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:55.667829 31616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:55.669070 31616 hybrid_clock.cc:648] HybridClock initialized: now 1786474195669027 us; error 67 us; skew 500 ppm
Aug 11 18:49:55 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474195, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:55.672495 31616 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:55.673743 31616 webserver.cc:460] Webserver started at http://127.27.115.254:36827/ using document root <none> and password file <none>
I20260811 18:49:55.674501 31616 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:55.674582 31616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:55.674845 31616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:55.677327 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "9d4d8c7480a04593a96af68ac5fa7162"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "6d0b69fc6f4f7168c3bba609cd1f9b60"
server_key_iv: "6c1ab6830b9988552fbd3f747d795f4b"
server_key_version: "encryptionkey@0"
I20260811 18:49:55.678061 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "9d4d8c7480a04593a96af68ac5fa7162"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "6d0b69fc6f4f7168c3bba609cd1f9b60"
server_key_iv: "6c1ab6830b9988552fbd3f747d795f4b"
server_key_version: "encryptionkey@0"
I20260811 18:49:55.682796 31616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:55.686233 31635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:55.687870 31616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:49:55.688066 31616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "9d4d8c7480a04593a96af68ac5fa7162"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "6d0b69fc6f4f7168c3bba609cd1f9b60"
server_key_iv: "6c1ab6830b9988552fbd3f747d795f4b"
server_key_version: "encryptionkey@0"
I20260811 18:49:55.688220 31616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:55.702947 31616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:55.706429 31616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:55.706686 31616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:55.715292 31616 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39333
I20260811 18:49:55.715655 31693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39333 every 8 connection(s)
I20260811 18:49:55.716475 31616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:55.719800 31694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:55.722172 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31616
I20260811 18:49:55.722375 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:55.722666 28111 external_mini_cluster.cc:1428] Setting key 472143d645655b42e9918c23e735b14a
I20260811 18:49:55.727188 31694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162: Bootstrap starting.
I20260811 18:49:55.730403 31694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:55.731475 31694 log.cc:824] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:49:55 dist-test-slave-9jjr krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:55.734364 31694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162: No bootstrap required, opened a new log
I20260811 18:49:55.739889 31694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } }
I20260811 18:49:55.740260 31694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:55.740319 31694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d4d8c7480a04593a96af68ac5fa7162, State: Initialized, Role: FOLLOWER
I20260811 18:49:55.740932 31694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } }
I20260811 18:49:55.741081 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:55.724463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.27.115.254:39333)
0811 18:49:55.724984 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:55.725003 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:55.725047 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:49:55.725886 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:55.725909 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:55.726394 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:55.726731 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:55.727793 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:55.728866 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:55.730243 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:55.730886 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:55.731037 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:55.734218 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:55.734226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:55.734246 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:55.734284 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:55.736560 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:55.737203 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:55.737214 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:55.737223 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:55.737424 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:55.738107 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:55.738111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:55.738113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:55.738841 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:55.739093 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:55.739524 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 18:49:55.739909 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":172,"threads_started":1}
I20260811 18:49:55.741123 31694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:55.741189 31694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:55.741286 31694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:55.742728 31694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } }
I20260811 18:49:55.743191 31694 leader_election.cc:302] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d4d8c7480a04593a96af68ac5fa7162; no voters:
I20260811 18:49:55.744189 31694 leader_election.cc:288] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:55.744302 31700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:55.744655 31700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [term 1 LEADER]: Becoming Leader. State: Replica: 9d4d8c7480a04593a96af68ac5fa7162, State: Running, Role: LEADER
I20260811 18:49:55.745048 31700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } }
I20260811 18:49:55.745932 31694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:55.747120 31702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d4d8c7480a04593a96af68ac5fa7162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } } }
I20260811 18:49:55.747118 31701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d4d8c7480a04593a96af68ac5fa7162. Latest consensus state: current_term: 1 leader_uuid: "9d4d8c7480a04593a96af68ac5fa7162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4d8c7480a04593a96af68ac5fa7162" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39333 } } }
I20260811 18:49:55.747316 31701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:55.747316 31702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:55.747917 31711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:55.751737 31711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:55.759447 31711 catalog_manager.cc:1347] Generated new cluster ID: 4382bfb5f60f489e98d4bc2c3bcb7fbc
I20260811 18:49:55.759565 31711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:55.773183 31711 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:55.774777 31711 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:55.785593 31711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d4d8c7480a04593a96af68ac5fa7162: Generated new TSK 0
I20260811 18:49:55.786590 31711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:55.831058 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:55.960273 31724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:55.960577 31724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:55.960625 31724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:55.964946 31724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:55.965039 31724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:55.965103 31724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:55.971012 31724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:55.972309 31724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:55.973464 31724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:55.984899 31733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:55.984908 31730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:55.984958 31731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:55.985268 31724 server_base.cc:1034] running on GCE node
I20260811 18:49:55.985769 31724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:55.986515 31724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:55.987679 31724 hybrid_clock.cc:648] HybridClock initialized: now 1786474195987660 us; error 35 us; skew 500 ppm
Aug 11 18:49:55 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474195, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:55.990833 31724 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:55.991940 31724 webserver.cc:460] Webserver started at http://127.27.115.193:38163/ using document root <none> and password file <none>
I20260811 18:49:55.992668 31724 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:55.992745 31724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:55.993034 31724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:55.995451 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "f027d3c234e947088f7005b76e80ab68"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "03b1fa23bcc86205cbbc91432e69023b"
server_key_iv: "2c2dc21a58f99ece6749fc83f53c0faa"
server_key_version: "encryptionkey@0"
I20260811 18:49:55.996054 31724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "f027d3c234e947088f7005b76e80ab68"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "03b1fa23bcc86205cbbc91432e69023b"
server_key_iv: "2c2dc21a58f99ece6749fc83f53c0faa"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.000207 31724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:49:56.003079 31741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.004403 31724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:49:56.004550 31724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "f027d3c234e947088f7005b76e80ab68"
format_stamp: "Formatted at 2026-08-11 18:49:55 on dist-test-slave-9jjr"
server_key: "03b1fa23bcc86205cbbc91432e69023b"
server_key_iv: "2c2dc21a58f99ece6749fc83f53c0faa"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.004685 31724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:56.032274 31724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:56.036293 31724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:56.036533 31724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:56.037287 31724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:56.038334 31724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:56.038390 31724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.038434 31724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:56.038450 31724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.050230 31724 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:36809
I20260811 18:49:56.050308 31872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:36809 every 8 connection(s)
I20260811 18:49:56.051596 31724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:56.060897 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31724
I20260811 18:49:56.061064 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:56.061632 28111 external_mini_cluster.cc:1428] Setting key 299bd00996e2482fe196bb6904432811
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474195, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:56.072293 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.054340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:35893 (local address 127.27.115.254:39333)
0811 18:49:56.054547 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:56.054553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:56.056284 (+ 1731us) server_negotiation.cc:407] Connection header received
0811 18:49:56.057595 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:56.057600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:56.057689 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:56.057799 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:56.059685 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.060366 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.062983 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.063481 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.063792 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.067664 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:56.067668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:56.067670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:56.067703 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:56.069377 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.070445 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.070448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.070450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.070546 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.070990 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.070994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.070996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.071225 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:56.071345 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:56.071947 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 18:49:56.072109 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:49:56.073274 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.054869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39333 (local address 127.27.115.193:35893)
0811 18:49:56.056035 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:56.056095 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:56.057222 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:56.058103 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:56.058114 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:56.058690 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:56.059368 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.059389 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.060630 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.060637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.062747 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.062768 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.063892 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.063900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.064234 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.065109 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:56.065140 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:56.067360 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:56.069551 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.069559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.069579 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.070297 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.070660 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.070664 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.070666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.070872 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.071346 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:56.071355 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:56.071674 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 18:49:56.071887 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 18:49:56.072290 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":168,"threads_started":1}
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:49:56.074780 31873 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39333
I20260811 18:49:56.075064 31873 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:56.075865 31873 heartbeater.cc:511] Master 127.27.115.254:39333 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:56.078081 31652 ts_manager.cc:194] Registered new tserver with Master: f027d3c234e947088f7005b76e80ab68 (127.27.115.193:36809)
I20260811 18:49:56.079617 31652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:35893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:56.098260 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:56.229656 31879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:56.229990 31879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:56.230038 31879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:56.234553 31879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:56.234637 31879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:56.234735 31879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:56.241034 31879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:56.242480 31879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:56.243780 31879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:56.255112 31889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:56.255103 31891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:56.255398 31887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:56.255491 31879 server_base.cc:1034] running on GCE node
I20260811 18:49:56.256335 31879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:56.257108 31879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:56.258291 31879 hybrid_clock.cc:648] HybridClock initialized: now 1786474196258258 us; error 47 us; skew 500 ppm
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:56.261931 31879 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:56.263159 31879 webserver.cc:460] Webserver started at http://127.27.115.194:40229/ using document root <none> and password file <none>
I20260811 18:49:56.263931 31879 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:56.264070 31879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:56.264345 31879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:56.267114 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "4490f7f1bae4451b8ded9fcb127c2728"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "78c5e574252687053483f98a4ced60c0"
server_key_iv: "64f409dab807a0a1423df43a22050132"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.267865 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "4490f7f1bae4451b8ded9fcb127c2728"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "78c5e574252687053483f98a4ced60c0"
server_key_iv: "64f409dab807a0a1423df43a22050132"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.272848 31879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:49:56.276145 31898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.277782 31879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:49:56.277961 31879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "4490f7f1bae4451b8ded9fcb127c2728"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "78c5e574252687053483f98a4ced60c0"
server_key_iv: "64f409dab807a0a1423df43a22050132"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.278105 31879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:56.297777 31879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:56.301601 31879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:56.301852 31879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:56.302690 31879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:56.303668 31879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:56.303741 31879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.303815 31879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:56.303857 31879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.315011 31879 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:40367
I20260811 18:49:56.315039 32033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:40367 every 8 connection(s)
I20260811 18:49:56.316088 31879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:56.317829 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 31879
I20260811 18:49:56.317930 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:56.318336 28111 external_mini_cluster.cc:1428] Setting key 52efcf5e0f0cad2f1ea9d3a066c74aea
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:49:56.337030 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.318261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37687 (local address 127.27.115.254:39333)
0811 18:49:56.318428 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:56.318432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:56.319704 (+ 1272us) server_negotiation.cc:407] Connection header received
0811 18:49:56.321767 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:56.321770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:56.321838 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:56.321930 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:56.323866 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.324715 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.327427 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.327687 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.327799 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.331667 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:56.331671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:56.331673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:56.331709 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:56.333661 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.334454 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.334464 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.334465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.334589 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.335213 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.335217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.335218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.335508 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:56.335651 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:56.336519 (+ 868us) server_negotiation.cc:299] Negotiation successful
0811 18:49:56.336755 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:49:56.337702 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.318779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39333 (local address 127.27.115.194:37687)
0811 18:49:56.319495 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:56.319553 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:56.321418 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:56.322235 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:56.322248 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:56.322899 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:56.323633 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.323653 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.324946 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.324953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.327176 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.327205 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.327924 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.327930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.328238 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.329069 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:56.329112 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:56.331371 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:56.333864 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.333872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.333890 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.334316 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.334782 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.334786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.334795 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.335007 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.335676 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:56.335687 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:56.336100 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 18:49:56.336301 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 18:49:56.336669 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":161,"threads_started":1}
I20260811 18:49:56.339370 32034 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39333
I20260811 18:49:56.339684 32034 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:56.340328 32034 heartbeater.cc:511] Master 127.27.115.254:39333 requested a full tablet report, sending...
I20260811 18:49:56.342443 31652 ts_manager.cc:194] Registered new tserver with Master: 4490f7f1bae4451b8ded9fcb127c2728 (127.27.115.194:40367)
I20260811 18:49:56.343308 31652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:37687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:56.359488 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:56.488446 32040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:56.488761 32040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:56.488808 32040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:56.493146 32040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:56.493247 32040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:56.493320 32040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:56.499265 32040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:56.500905 32040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:56.501989 32040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:56.512429 32049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:56.512445 32046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:56.512445 32047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:56.512755 32040 server_base.cc:1034] running on GCE node
I20260811 18:49:56.513208 32040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:56.513924 32040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:56.515141 32040 hybrid_clock.cc:648] HybridClock initialized: now 1786474196515119 us; error 42 us; skew 500 ppm
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:56.517962 32040 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:56.519047 32040 webserver.cc:460] Webserver started at http://127.27.115.195:38397/ using document root <none> and password file <none>
I20260811 18:49:56.519667 32040 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:56.519769 32040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:56.520015 32040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:56.522274 32040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "2197ccda1b1148bcb8a6e5e1bb14f92a"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "71bea08517e9d43878b73c6b096a6048"
server_key_iv: "cd3bc9b6891c885b602e324ee26bd39c"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.522873 32040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "2197ccda1b1148bcb8a6e5e1bb14f92a"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "71bea08517e9d43878b73c6b096a6048"
server_key_iv: "cd3bc9b6891c885b602e324ee26bd39c"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.526896 32040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:49:56.529600 32056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.531077 32040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:49:56.531272 32040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2197ccda1b1148bcb8a6e5e1bb14f92a"
format_stamp: "Formatted at 2026-08-11 18:49:56 on dist-test-slave-9jjr"
server_key: "71bea08517e9d43878b73c6b096a6048"
server_key_iv: "cd3bc9b6891c885b602e324ee26bd39c"
server_key_version: "encryptionkey@0"
I20260811 18:49:56.531410 32040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:56.553399 32040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:56.556933 32040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:56.557158 32040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:56.557835 32040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:56.558808 32040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:56.558881 32040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.558955 32040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:56.559006 32040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:56.568374 32040 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:34691
I20260811 18:49:56.568401 32184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:34691 every 8 connection(s)
I20260811 18:49:56.569304 32040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:56.578428 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 32040
I20260811 18:49:56.578598 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:56.578923 28111 external_mini_cluster.cc:1428] Setting key 5b948aaf3dc3fe12529d164123404a62
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:56.584933 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.571476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:47003 (local address 127.27.115.254:39333)
0811 18:49:56.571651 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:56.571655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:56.572538 (+ 883us) server_negotiation.cc:407] Connection header received
0811 18:49:56.574012 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:56.574016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:56.574079 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:56.574223 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:56.575483 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.576031 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.577202 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.577427 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.577545 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.580813 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:56.580817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:56.580820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:56.580849 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:56.582593 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.583254 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.583258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.583260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.583350 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.583738 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.583741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.583743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.583945 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:56.584061 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:56.584613 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 18:49:56.584764 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:49:56.585834 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.571859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39333 (local address 127.27.115.195:47003)
0811 18:49:56.572364 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:56.572419 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:56.573784 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:56.574411 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:56.574418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:56.574816 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:56.575318 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.575332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.576179 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.576187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.577054 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:56.577064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.577597 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.577601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:56.577777 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.578539 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:56.578556 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:56.580641 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:56.582765 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.582772 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.582790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.583117 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.583452 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:56.583456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:56.583458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:56.583619 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:56.584061 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:56.584070 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:56.584360 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 18:49:56.584570 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 18:49:56.584934 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":125,"threads_started":1}
I20260811 18:49:56.587147 32185 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39333
I20260811 18:49:56.587414 32185 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:56.587960 32185 heartbeater.cc:511] Master 127.27.115.254:39333 requested a full tablet report, sending...
I20260811 18:49:56.589110 31652 ts_manager.cc:194] Registered new tserver with Master: 2197ccda1b1148bcb8a6e5e1bb14f92a (127.27.115.195:34691)
I20260811 18:49:56.589747 31652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:47003
I20260811 18:49:56.593266 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474196, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:56.621754 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:56.612784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.27.115.254:39333)
0811 18:49:56.612923 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:56.612927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:56.613034 (+ 107us) server_negotiation.cc:407] Connection header received
0811 18:49:56.613136 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:56.613138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:56.613208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:56.613293 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:56.614025 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.614643 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.615617 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:56.615878 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:56.615978 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:56.618607 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:56.618611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:56.618613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:56.618643 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:56.620046 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.620504 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.620507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.620508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.620587 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:56.620842 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:56.620844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:56.620845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:56.621079 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:56.621217 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:56.621505 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:49:56.621623 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:49:56.622288 31652 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60058
I20260811 18:49:56.623255 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31724
I20260811 18:49:56.634681 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31879
I20260811 18:49:56.650004 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 32040
I20260811 18:49:56.659705 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 31616
2026-08-11T18:49:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2440 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[32198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[32198](info): set up 2 sockets
Aug 11 18:49:56 dist-test-slave-9jjr krb5kdc[32198](info): commencing operation
krb5kdc: starting...
W20260811 18:49:57.796885 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:49:57 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:49:57.942444 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39737
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:58.065259 32211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:58.065640 32211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:58.065734 32211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:58.070111 32211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:58.070269 32211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:58.070307 32211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:58.070346 32211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:58.070380 32211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:58.076296 32211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:58.077610 32211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:58.078769 32211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:58.087994 32220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.088004 32217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.088204 32211 server_base.cc:1034] running on GCE node
W20260811 18:49:58.088006 32216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.088982 32211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:58.090255 32211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:58.091449 32211 hybrid_clock.cc:648] HybridClock initialized: now 1786474198091427 us; error 35 us; skew 500 ppm
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:58.098696 32211 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:49:58.099822 32211 webserver.cc:460] Webserver started at http://127.27.115.254:42101/ using document root <none> and password file <none>
I20260811 18:49:58.100419 32211 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:58.100512 32211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:58.100762 32211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:58.103127 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "ce6edaedc50a4090bf3fba42ddcad907"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "e4e3035df2e61b3c980d2986e461a828"
server_key_iv: "08c8c426e88f44470c5d0f147336f831"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.103765 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "ce6edaedc50a4090bf3fba42ddcad907"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "e4e3035df2e61b3c980d2986e461a828"
server_key_iv: "08c8c426e88f44470c5d0f147336f831"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.108044 32211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 18:49:58.111003 32228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.112567 32211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:49:58.112977 32211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "ce6edaedc50a4090bf3fba42ddcad907"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "e4e3035df2e61b3c980d2986e461a828"
server_key_iv: "08c8c426e88f44470c5d0f147336f831"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.113162 32211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:58.141687 32211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:58.145910 32211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:58.146329 32211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:58.155344 32211 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39737
I20260811 18:49:58.155371 32289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39737 every 8 connection(s)
I20260811 18:49:58.156466 32211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:58.159777 32290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:58.162317 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 32211
I20260811 18:49:58.162468 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:49:58.162830 28111 external_mini_cluster.cc:1428] Setting key cec92977d8cc3116b22703acce4b8202
I20260811 18:49:58.167873 32290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907: Bootstrap starting.
I20260811 18:49:58.171373 32290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:58.172204 32290 log.cc:824] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:58.174719 32290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907: No bootstrap required, opened a new log
I20260811 18:49:58.179615 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:58.164440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.27.115.254:39737)
0811 18:49:58.164914 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:58.164931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:58.164969 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:49:58.165633 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:58.165650 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:58.165987 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:58.166346 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:58.167334 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.168465 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.169854 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.170675 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.170839 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:58.173836 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:58.173843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:58.173860 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:58.173891 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:58.176013 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.176496 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.176503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.176509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.176671 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.177003 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.177009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.177013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.177675 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:58.177864 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:58.178200 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 18:49:58.178532 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":153,"threads_started":1}
I20260811 18:49:58.179734 32290 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } }
I20260811 18:49:58.180012 32290 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:58.180080 32290 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6edaedc50a4090bf3fba42ddcad907, State: Initialized, Role: FOLLOWER
I20260811 18:49:58.180531 32290 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } }
I20260811 18:49:58.180703 32290 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:58.180799 32290 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:58.180912 32290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:58.182248 32290 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } }
I20260811 18:49:58.182623 32290 leader_election.cc:302] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce6edaedc50a4090bf3fba42ddcad907; no voters:
I20260811 18:49:58.183025 32290 leader_election.cc:288] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:58.183151 32297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:58.183411 32297 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [term 1 LEADER]: Becoming Leader. State: Replica: ce6edaedc50a4090bf3fba42ddcad907, State: Running, Role: LEADER
I20260811 18:49:58.183826 32297 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } }
I20260811 18:49:58.184378 32290 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:58.185671 32298 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce6edaedc50a4090bf3fba42ddcad907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } } }
I20260811 18:49:58.185678 32299 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce6edaedc50a4090bf3fba42ddcad907. Latest consensus state: current_term: 1 leader_uuid: "ce6edaedc50a4090bf3fba42ddcad907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce6edaedc50a4090bf3fba42ddcad907" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39737 } } }
I20260811 18:49:58.185830 32298 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:58.185832 32299 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:58.186249 32306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:58.189692 32306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:58.195495 32306 catalog_manager.cc:1347] Generated new cluster ID: 161563519ac64c8ab9c641d1cf551e38
I20260811 18:49:58.195585 32306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:58.222582 32306 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:58.223681 32306 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:58.232539 32306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce6edaedc50a4090bf3fba42ddcad907: Generated new TSK 0
I20260811 18:49:58.233381 32306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:49:58.274951 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:58.408437 32319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:58.408809 32319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:58.408864 32319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:58.413959 32319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:58.414057 32319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:58.414212 32319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:49:58.420758 32319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:58.422108 32319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:58.423415 32319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:58.434274 32327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.434269 32324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.434437 32319 server_base.cc:1034] running on GCE node
W20260811 18:49:58.434269 32325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.435151 32319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:58.435918 32319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:58.437134 32319 hybrid_clock.cc:648] HybridClock initialized: now 1786474198437119 us; error 39 us; skew 500 ppm
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:58.440830 32319 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:49:58.442078 32319 webserver.cc:460] Webserver started at http://127.27.115.193:42085/ using document root <none> and password file <none>
I20260811 18:49:58.442828 32319 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:58.442904 32319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:58.443181 32319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:58.445726 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "a0bbbf3614d947cf92dcfd062be0708a"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "252e94e7e3909b2e114072db0241fcea"
server_key_iv: "6e5d307dbc3293fb5dd3786c3fd3e514"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.446489 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "a0bbbf3614d947cf92dcfd062be0708a"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "252e94e7e3909b2e114072db0241fcea"
server_key_iv: "6e5d307dbc3293fb5dd3786c3fd3e514"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.451121 32319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:49:58.454475 32334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.456000 32319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:49:58.456132 32319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "a0bbbf3614d947cf92dcfd062be0708a"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "252e94e7e3909b2e114072db0241fcea"
server_key_iv: "6e5d307dbc3293fb5dd3786c3fd3e514"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.456247 32319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:58.482345 32319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:58.486702 32319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:58.486894 32319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:58.487541 32319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:58.488664 32319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:58.488719 32319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.488766 32319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:58.488781 32319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.499308 32319 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:42471
I20260811 18:49:58.499356 32466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:42471 every 8 connection(s)
I20260811 18:49:58.500663 32319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:58.505184 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 32319
I20260811 18:49:58.505363 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:58.505729 28111 external_mini_cluster.cc:1428] Setting key 0f04becdc9bab1043b6a58f1286bd6c0
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:49:58.521936 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:58.503075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:58417 (local address 127.27.115.254:39737)
0811 18:49:58.503273 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:58.503279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:58.504862 (+ 1583us) server_negotiation.cc:407] Connection header received
0811 18:49:58.507819 (+ 2957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:58.507823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:58.507875 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:58.507962 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:58.510066 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.511224 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.513386 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.513660 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.513776 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:58.517674 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:58.517693 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:58.517696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:58.517727 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:58.519457 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.520133 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.520137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.520139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.520242 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.520653 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.520656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.520658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.520875 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:58.521001 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:58.521590 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 18:49:58.521757 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:49:58.523150 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:58.503730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39737 (local address 127.27.115.193:58417)
0811 18:49:58.504586 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:58.504673 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:58.507485 (+ 2812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:58.508238 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:58.508253 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:58.508942 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:58.509791 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:58.509822 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.511438 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.511441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:58.513064 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:58.513084 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.513895 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.513902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:58.514236 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:58.515047 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:58.515079 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:58.517361 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:58.519617 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:58.519625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:58.519645 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:58.519950 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:58.520355 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:58.520358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:58.520361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:58.520533 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:58.521001 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:58.521011 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:58.521317 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 18:49:58.521544 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:49:58.521930 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":203,"threads_started":1}
I20260811 18:49:58.524720 32467 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39737
I20260811 18:49:58.525010 32467 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:58.525588 32467 heartbeater.cc:511] Master 127.27.115.254:39737 requested a full tablet report, sending...
I20260811 18:49:58.527686 32246 ts_manager.cc:194] Registered new tserver with Master: a0bbbf3614d947cf92dcfd062be0708a (127.27.115.193:42471)
I20260811 18:49:58.528957 32246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:58417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:49:58.542544 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:58.678753 32474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:58.679324 32474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:58.679519 32474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:58.685190 32474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:58.685281 32474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:58.685393 32474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:49:58.691609 32474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:58.692771 32474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:58.693826 32474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:58.705281 32480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.705298 32479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.705282 32482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.705538 32474 server_base.cc:1034] running on GCE node
I20260811 18:49:58.706348 32474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:58.707190 32474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:58.708424 32474 hybrid_clock.cc:648] HybridClock initialized: now 1786474198708398 us; error 44 us; skew 500 ppm
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:58.711932 32474 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:49:58.713029 32474 webserver.cc:460] Webserver started at http://127.27.115.194:33113/ using document root <none> and password file <none>
I20260811 18:49:58.713687 32474 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:58.713788 32474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:58.714054 32474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:58.716526 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "dbff10704d204c30a2867889a2770e99"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "ba5cab70d5799a78afbc814319d22e0c"
server_key_iv: "8b00f3cb4d72075efe6d155855caccbd"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.717305 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "dbff10704d204c30a2867889a2770e99"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "ba5cab70d5799a78afbc814319d22e0c"
server_key_iv: "8b00f3cb4d72075efe6d155855caccbd"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.722600 32474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:49:58.726915 32489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.729050 32474 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:49:58.729561 32474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "dbff10704d204c30a2867889a2770e99"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "ba5cab70d5799a78afbc814319d22e0c"
server_key_iv: "8b00f3cb4d72075efe6d155855caccbd"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.729779 32474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:58.760613 32474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:58.765195 32474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:58.765468 32474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:58.766283 32474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:58.767326 32474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:58.767405 32474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.767490 32474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:58.767542 32474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.778031 32474 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:34287
I20260811 18:49:58.778066 32618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:34287 every 8 connection(s)
I20260811 18:49:58.779166 32474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:58.782317 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 32474
I20260811 18:49:58.782460 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:58.782823 28111 external_mini_cluster.cc:1428] Setting key 9076815aff53b0528596ab6933f80426
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:49:58.796552 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:58.781474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:58427 (local address 127.27.115.254:39737)
0811 18:49:58.781646 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:58.781650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:58.782831 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 18:49:58.784614 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:58.784617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:58.784685 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:58.784771 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:58.786296 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.786996 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.788401 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.788644 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.788772 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:58.792072 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:58.792076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:58.792078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:58.792109 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:58.793755 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.794534 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.794537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.794539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.794637 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:58.795071 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:58.795075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:58.795077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:58.795331 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:58.795455 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:58.796140 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 18:49:58.796348 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:49:58.797570 32621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:58.781846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39737 (local address 127.27.115.194:58427)
0811 18:49:58.782622 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:58.782679 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:58.784356 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:58.785018 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:58.785028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:58.785464 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:58.786077 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:58.786094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.787160 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.787164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:58.788253 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:58.788265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:58.788833 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:58.788837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:58.789026 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:58.789782 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:58.789802 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:58.791834 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:58.793910 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:58.793917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:58.793937 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:58.794392 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:58.794739 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:58.794759 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:58.794761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:58.794943 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:58.795456 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:58.795465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:58.795796 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 18:49:58.796020 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 18:49:58.796477 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":235,"threads_started":1}
I20260811 18:49:58.799311 32619 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39737
I20260811 18:49:58.799688 32619 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:58.800374 32619 heartbeater.cc:511] Master 127.27.115.254:39737 requested a full tablet report, sending...
I20260811 18:49:58.801901 32246 ts_manager.cc:194] Registered new tserver with Master: dbff10704d204c30a2867889a2770e99 (127.27.115.194:34287)
I20260811 18:49:58.802646 32246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:58427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:49:58.818048 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:58.946297 32625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:58.946647 32625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:58.946728 32625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:58.951431 32625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:58.951583 32625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:58.951689 32625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:49:58.957746 32625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:49:58.959313 32625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:58.960569 32625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:58.971383 32634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.971382 32631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:58.971453 32632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:58.971760 32625 server_base.cc:1034] running on GCE node
I20260811 18:49:58.972321 32625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:58.973073 32625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:58.974273 32625 hybrid_clock.cc:648] HybridClock initialized: now 1786474198974234 us; error 54 us; skew 500 ppm
Aug 11 18:49:58 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:58.977787 32625 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:49:58.979012 32625 webserver.cc:460] Webserver started at http://127.27.115.195:38921/ using document root <none> and password file <none>
I20260811 18:49:58.979797 32625 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:58.979934 32625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:58.980234 32625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:49:58.983014 32625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "2a2e3c13f4f34c7cb086ea091be7b306"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "d643989f03d134dbbe311524af8bd141"
server_key_iv: "f6717ceec37d5e5ff4a3fbb628ce6e7b"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.983777 32625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "2a2e3c13f4f34c7cb086ea091be7b306"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "d643989f03d134dbbe311524af8bd141"
server_key_iv: "f6717ceec37d5e5ff4a3fbb628ce6e7b"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.989012 32625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:49:58.992254 32642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:58.993749 32625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:49:58.993950 32625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2a2e3c13f4f34c7cb086ea091be7b306"
format_stamp: "Formatted at 2026-08-11 18:49:58 on dist-test-slave-9jjr"
server_key: "d643989f03d134dbbe311524af8bd141"
server_key_iv: "f6717ceec37d5e5ff4a3fbb628ce6e7b"
server_key_version: "encryptionkey@0"
I20260811 18:49:58.994115 32625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:59.020623 32625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:59.024252 32625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:59.024506 32625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:59.025226 32625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:59.026237 32625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:59.026316 32625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:59.026410 32625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:59.026466 32625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:59.038064 32625 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:36325
I20260811 18:49:59.038305 313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:36325 every 8 connection(s)
I20260811 18:49:59.039377 32625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:59.047580 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 32625
I20260811 18:49:59.047883 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:59.048291 28111 external_mini_cluster.cc:1428] Setting key fc69b2b529fb1ef1941b3f0e85a1fb6b
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474198, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:59.058328 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:59.041826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:56151 (local address 127.27.115.254:39737)
0811 18:49:59.041999 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:59.042003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:59.043338 (+ 1335us) server_negotiation.cc:407] Connection header received
0811 18:49:59.044990 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:59.044994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:59.045073 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:59.045195 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:59.046812 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.047575 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.049382 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.049649 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.049792 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:59.053542 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:59.053546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:59.053548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:59.053582 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:59.055575 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.056329 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.056333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.056334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.056431 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.056879 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.056885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.056891 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.057125 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:59.057255 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:59.057900 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 18:49:59.058077 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:49:59.059314 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:59.042301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39737 (local address 127.27.115.195:56151)
0811 18:49:59.043109 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:59.043177 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:59.044697 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:59.045459 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:59.045468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:59.045930 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:59.046609 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:59.046628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.047793 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.047797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:59.049217 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:59.049243 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.049863 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.049868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:59.050106 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:59.051386 (+ 1280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:59.051407 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:59.053270 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:59.055765 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:59.055775 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:59.055804 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:59.056172 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:59.056546 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:59.056550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:59.056552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:59.056748 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:59.057257 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:59.057269 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:59.057590 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 18:49:59.057830 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 18:49:59.058283 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":188,"threads_started":1}
I20260811 18:49:59.060761 314 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39737
I20260811 18:49:59.061314 314 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:59.061988 314 heartbeater.cc:511] Master 127.27.115.254:39737 requested a full tablet report, sending...
I20260811 18:49:59.063617 32246 ts_manager.cc:194] Registered new tserver with Master: 2a2e3c13f4f34c7cb086ea091be7b306 (127.27.115.195:36325)
I20260811 18:49:59.064188 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:49:59.064563 32246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:56151
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[32198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:49:59.096927 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:59.085806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.27.115.193:42471)
0811 18:49:59.086203 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:59.086212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:59.086266 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:49:59.086323 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:59.086328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:59.086598 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:59.086727 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:59.087462 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.088249 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.089181 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.089642 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.089775 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:59.092808 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:59.092824 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:59.092843 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:59.092891 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:59.094880 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.095384 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.095390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.095391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.095484 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.095802 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.095804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.095806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.096082 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:59.096260 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:59.096509 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:49:59.096701 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":122,"threads_started":1}
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[32198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:49:59.111078 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:59.100441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.27.115.254:39737)
0811 18:49:59.100588 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:59.100595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:59.100700 (+ 105us) server_negotiation.cc:407] Connection header received
0811 18:49:59.100889 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:59.100892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:59.100938 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:59.101028 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:59.101762 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.102392 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.103294 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:59.103526 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:59.103643 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:59.106669 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:59.106673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:59.106676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:59.106714 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:59.108329 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.108972 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.108976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.108978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.109090 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:59.109607 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:59.109619 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:59.109621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:59.110100 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:59.110418 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:59.110776 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 18:49:59.110935 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 18:49:59.111671 32246 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45764
I20260811 18:49:59.112627 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 32319
I20260811 18:49:59.125674 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 32474
I20260811 18:49:59.133524 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 32625
I20260811 18:49:59.149027 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 32211
2026-08-11T18:49:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2488 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[327](info): set up 2 sockets
Aug 11 18:49:59 dist-test-slave-9jjr krb5kdc[327](info): commencing operation
krb5kdc: starting...
W20260811 18:50:00.299249 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:00 dist-test-slave-9jjr krb5kdc[327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:50:00.447683 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45721
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:00.571770 342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:00.572093 342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:00.572139 342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:00.576965 342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:00.577055 342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:00.577076 342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:00.577095 342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:00.577111 342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:00.582928 342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:00.584192 342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:00.585415 342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:00.594405 350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:00.594450 347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:00.594246 348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:00.594854 342 server_base.cc:1034] running on GCE node
I20260811 18:50:00.595613 342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:00.596753 342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:00.597926 342 hybrid_clock.cc:648] HybridClock initialized: now 1786474200597900 us; error 42 us; skew 500 ppm
Aug 11 18:50:00 dist-test-slave-9jjr krb5kdc[327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474200, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:00.600909 342 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:50:00.601680 342 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:00.601748 342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:00.601928 342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:00.604130 342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "043735d0419a4bb48f81a5e32abe1631"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "877d90a7b11a7796690a3467d636a605"
server_key_iv: "75ce94f8eda7085be7159bd7d70f595e"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.604691 342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "043735d0419a4bb48f81a5e32abe1631"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "877d90a7b11a7796690a3467d636a605"
server_key_iv: "75ce94f8eda7085be7159bd7d70f595e"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.608724 342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:00.611819 358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:00.613516 342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:50:00.613669 342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "043735d0419a4bb48f81a5e32abe1631"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "877d90a7b11a7796690a3467d636a605"
server_key_iv: "75ce94f8eda7085be7159bd7d70f595e"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.613795 342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:00.626375 342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:00.629876 342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:00.630053 342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:00.638033 342 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:45721
I20260811 18:50:00.638371 423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:45721 every 8 connection(s)
I20260811 18:50:00.638896 342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:00.642223 424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:00.646378 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 342
I20260811 18:50:00.646539 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:00.646849 28111 external_mini_cluster.cc:1428] Setting key ad57ba8d9b305dbc43201e4dfc1c8c2f
I20260811 18:50:00.649448 424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: Bootstrap starting.
I20260811 18:50:00.652263 424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:00.653244 424 log.cc:824] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:00.655879 424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: No bootstrap required, opened a new log
Aug 11 18:50:00 dist-test-slave-9jjr krb5kdc[327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:00.661191 424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } }
I20260811 18:50:00.661589 424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:00.661679 424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043735d0419a4bb48f81a5e32abe1631, State: Initialized, Role: FOLLOWER
I20260811 18:50:00.662369 424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } }
I20260811 18:50:00.662586 424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:00.662671 424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:00.662787 424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:00.664264 424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } }
I20260811 18:50:00.664731 424 leader_election.cc:302] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043735d0419a4bb48f81a5e32abe1631; no voters:
I20260811 18:50:00.665179 424 leader_election.cc:288] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:00.665337 429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:00.665598 429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [term 1 LEADER]: Becoming Leader. State: Replica: 043735d0419a4bb48f81a5e32abe1631, State: Running, Role: LEADER
I20260811 18:50:00.665995 429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } }
I20260811 18:50:00.666443 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:00.648414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.27.115.254:45721)
0811 18:50:00.648878 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:00.648895 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:00.648932 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:50:00.649691 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:00.649709 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:00.650054 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:00.650415 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:00.651382 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.652527 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.653743 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.654462 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.654601 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:00.657773 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:00.657784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:00.657808 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:00.657861 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:00.661643 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:00.662497 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:00.662506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:00.662514 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:00.662725 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:00.663161 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:00.663165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:00.663167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:00.663793 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:00.664024 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:00.664938 (+ 914us) server_negotiation.cc:299] Negotiation successful
0811 18:50:00.665313 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":144,"threads_started":1}
I20260811 18:50:00.666661 424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:00.669178 431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043735d0419a4bb48f81a5e32abe1631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } } }
I20260811 18:50:00.669502 431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:00.670436 432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043735d0419a4bb48f81a5e32abe1631. Latest consensus state: current_term: 1 leader_uuid: "043735d0419a4bb48f81a5e32abe1631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043735d0419a4bb48f81a5e32abe1631" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45721 } } }
I20260811 18:50:00.670559 432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631 [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:00.672580 448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:00.672688 448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:00.672819 451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:00.674445 451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:00.681177 451 catalog_manager.cc:1347] Generated new cluster ID: d82490b427d94c6e8cde4320e065231f
I20260811 18:50:00.681279 451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:00.703088 451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:00.704298 451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:00.713130 451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: Generated new TSK 0
I20260811 18:50:00.714389 451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:50:00.759341 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:00.906740 457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:00.907095 457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:00.907150 457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:00.911639 457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:00.911727 457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:00.911795 457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:00.917697 457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:00.919066 457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:00.920241 457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:00.931109 464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:00.931111 465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:00.931109 467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:00.931509 457 server_base.cc:1034] running on GCE node
I20260811 18:50:00.932076 457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:00.932942 457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:00.934257 457 hybrid_clock.cc:648] HybridClock initialized: now 1786474200934229 us; error 33 us; skew 500 ppm
Aug 11 18:50:00 dist-test-slave-9jjr krb5kdc[327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474200, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:00.937547 457 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:50:00.938619 457 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:00.938740 457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:00.939013 457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:00.942041 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "25642519c5834e90a44eb9d3424b6c71"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "789f417bb206e387fb2977942b1110a1"
server_key_iv: "a5139008069d5984f856ca7ebc8b2e17"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.942888 457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "25642519c5834e90a44eb9d3424b6c71"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "789f417bb206e387fb2977942b1110a1"
server_key_iv: "a5139008069d5984f856ca7ebc8b2e17"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.947530 457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:00.950907 473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:00.952642 457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:00.952894 457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "25642519c5834e90a44eb9d3424b6c71"
format_stamp: "Formatted at 2026-08-11 18:50:00 on dist-test-slave-9jjr"
server_key: "789f417bb206e387fb2977942b1110a1"
server_key_iv: "a5139008069d5984f856ca7ebc8b2e17"
server_key_version: "encryptionkey@0"
I20260811 18:50:00.953070 457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:00.974243 457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:00.977738 457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:00.978080 457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:00.978816 457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:00.979816 457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:00.979893 457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:00.979970 457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:00.980022 457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:00.990772 457 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:46657
I20260811 18:50:00.990797 611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:46657 every 8 connection(s)
I20260811 18:50:00.991576 457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:01.000823 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 457
I20260811 18:50:01.001003 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:01.001319 28111 external_mini_cluster.cc:1428] Setting key 52b56b51982cc9add1035dbe013b3a8b
Aug 11 18:50:00 dist-test-slave-9jjr krb5kdc[327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474200, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:01.006780 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:00.993522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:45195 (local address 127.27.115.254:45721)
0811 18:50:00.993664 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:00.993668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:00.994600 (+ 932us) server_negotiation.cc:407] Connection header received
0811 18:50:00.996018 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:00.996022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:00.996082 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:00.996201 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:00.997425 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.997975 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.999220 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.999451 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.999590 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.002580 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:01.002584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:01.002587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:01.002611 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:01.004353 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.004988 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.004992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.004993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.005087 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.005468 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.005472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.005474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.005692 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:01.005813 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.006450 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.006599 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:50:01.007682 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:00.993892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45721 (local address 127.27.115.193:45195)
0811 18:50:00.994440 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:00.994489 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:00.995797 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:00.996377 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:00.996385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:00.996769 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:00.997273 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:00.997287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.998110 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.998119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:00.999077 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:00.999087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:00.999653 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:00.999657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:00.999837 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.000525 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:01.000542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:01.002398 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:01.004514 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.004521 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.004541 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.004864 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.005186 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.005190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.005192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.005355 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.005813 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:01.005822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:01.006111 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.006392 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.006774 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":190,"threads_started":1}
I20260811 18:50:01.009274 612 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45721
I20260811 18:50:01.009567 612 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:01.010354 612 heartbeater.cc:511] Master 127.27.115.254:45721 requested a full tablet report, sending...
I20260811 18:50:01.012154 382 ts_manager.cc:194] Registered new tserver with Master: 25642519c5834e90a44eb9d3424b6c71 (127.27.115.193:46657)
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:01.013417 382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:45195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:50:01.037671 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:01.163978 617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:01.164347 617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:01.164400 617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:01.169409 617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:01.169520 617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:01.169626 617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:01.175769 617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:01.177088 617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:01.178411 617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:01.189692 626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:01.189703 624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:01.189841 617 server_base.cc:1034] running on GCE node
W20260811 18:50:01.189692 623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:01.190639 617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:01.191381 617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:01.192621 617 hybrid_clock.cc:648] HybridClock initialized: now 1786474201192598 us; error 32 us; skew 500 ppm
Aug 11 18:50:01 dist-test-slave-9jjr krb5kdc[327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474201, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:01.195950 617 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:50:01.196867 617 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:01.196971 617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:01.197227 617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:01.199756 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "4999c14466544bb18813700132247503"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "bc3050d47d29688105d2cefc81a477df"
server_key_iv: "57fd052147e6ef01dd647ef47fe24cc4"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.200462 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "4999c14466544bb18813700132247503"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "bc3050d47d29688105d2cefc81a477df"
server_key_iv: "57fd052147e6ef01dd647ef47fe24cc4"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.205165 617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:01.208626 632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.210827 617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:01.211027 617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "4999c14466544bb18813700132247503"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "bc3050d47d29688105d2cefc81a477df"
server_key_iv: "57fd052147e6ef01dd647ef47fe24cc4"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.211186 617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:01.236948 617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:01.240927 617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:01.241170 617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:01.241847 617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:01.242945 617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:01.243023 617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.243117 617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:01.243161 617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.255306 617 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:34341
I20260811 18:50:01.255393 756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:34341 every 8 connection(s)
I20260811 18:50:01.256143 617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:01.257272 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 617
I20260811 18:50:01.257408 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:01.257797 28111 external_mini_cluster.cc:1428] Setting key 961a7afe570342ab2ff8e4d6ab8e5df5
Aug 11 18:50:01 dist-test-slave-9jjr krb5kdc[327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474201, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:50:01.276558 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.258769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:40651 (local address 127.27.115.254:45721)
0811 18:50:01.258928 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.258931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.260343 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 18:50:01.261976 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.261984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.262067 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.262229 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:01.264335 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.265097 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.266874 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.267138 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.267272 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.270863 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:01.270867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:01.270869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:01.270904 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:01.273247 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.274036 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.274044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.274048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.274244 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.274755 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.274761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.274766 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.275081 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:01.275214 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.276095 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.276289 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:01.277738 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.259474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45721 (local address 127.27.115.194:40651)
0811 18:50:01.260102 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.260169 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.261648 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.262709 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.262721 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.263353 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:01.264099 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.264117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.265379 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.265383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.266707 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.266722 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.267341 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.267346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.267620 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.268509 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:01.268538 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:01.270497 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:01.273461 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.273469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.273488 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.273873 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.274365 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.274371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.274376 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.274620 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.275304 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:01.275315 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:01.275755 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.276000 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.276421 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":190,"threads_started":1}
I20260811 18:50:01.279368 758 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45721
I20260811 18:50:01.279666 758 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:01.280382 758 heartbeater.cc:511] Master 127.27.115.254:45721 requested a full tablet report, sending...
I20260811 18:50:01.281751 382 ts_manager.cc:194] Registered new tserver with Master: 4999c14466544bb18813700132247503 (127.27.115.194:34341)
I20260811 18:50:01.282512 382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:40651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:50:01.297649 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:01.423805 764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:01.424115 764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:01.424175 764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:01.428478 764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:01.428563 764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:01.428629 764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:01.435107 764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:01.436334 764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:01.437495 764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:01.450317 773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:01.450317 772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:01.450317 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:01.450781 764 server_base.cc:1034] running on GCE node
I20260811 18:50:01.451407 764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:01.452566 764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:01.453819 764 hybrid_clock.cc:648] HybridClock initialized: now 1786474201453796 us; error 57 us; skew 500 ppm
Aug 11 18:50:01 dist-test-slave-9jjr krb5kdc[327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474201, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:01.457465 764 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:50:01.458590 764 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:01.458667 764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:01.458918 764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:01.461215 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "9e5dd7d874a8404ebc6d0259e0e7755c"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "0c50052aa91e8df130d9f5ee08f9ad87"
server_key_iv: "a5a3946fc2c57b03614655d473016cd7"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.461824 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "9e5dd7d874a8404ebc6d0259e0e7755c"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "0c50052aa91e8df130d9f5ee08f9ad87"
server_key_iv: "a5a3946fc2c57b03614655d473016cd7"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.465955 764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:50:01.469064 782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.470541 764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:01.470685 764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "9e5dd7d874a8404ebc6d0259e0e7755c"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "0c50052aa91e8df130d9f5ee08f9ad87"
server_key_iv: "a5a3946fc2c57b03614655d473016cd7"
server_key_version: "encryptionkey@0"
I20260811 18:50:01.470813 764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:01.494310 764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:01.497907 764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:01.498203 764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:01.498901 764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:01.499856 764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:01.499905 764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.499975 764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:01.500010 764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:01.511617 764 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:45179
I20260811 18:50:01.511627 913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:45179 every 8 connection(s)
I20260811 18:50:01.512434 764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:01.516790 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 764
I20260811 18:50:01.516968 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:01.517333 28111 external_mini_cluster.cc:1428] Setting key 267a2f008334a7db1af3dfc422d387ad
Aug 11 18:50:01 dist-test-slave-9jjr krb5kdc[327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474201, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:01.529695 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.514859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:42745 (local address 127.27.115.254:45721)
0811 18:50:01.515030 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.515035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.516158 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 18:50:01.517922 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.517925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.517986 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.518109 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:01.519957 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.520615 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.522002 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.522377 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.522484 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.525644 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:01.525648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:01.525650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:01.525682 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:01.527315 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.528012 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.528015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.528016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.528104 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.528467 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.528469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.528470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.528680 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:01.528770 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.529436 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.529562 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:50:01.530836 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.515423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45721 (local address 127.27.115.195:42745)
0811 18:50:01.515986 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.516036 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.517658 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.518579 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.518603 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.519113 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:01.519761 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.519786 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.520787 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.520791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.521847 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.521859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.522591 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.522595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.522782 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.523519 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:01.523541 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:01.525408 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:01.527487 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.527498 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.527524 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.527907 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.528208 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:01.528210 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:01.528212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:01.528375 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:01.528818 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:01.528825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:01.529140 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.529369 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.529684 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":140,"threads_started":1}
I20260811 18:50:01.532348 915 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45721
I20260811 18:50:01.532590 915 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:01.533061 915 heartbeater.cc:511] Master 127.27.115.254:45721 requested a full tablet report, sending...
I20260811 18:50:01.534271 381 ts_manager.cc:194] Registered new tserver with Master: 9e5dd7d874a8404ebc6d0259e0e7755c (127.27.115.195:45179)
I20260811 18:50:01.535112 381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:42745
I20260811 18:50:01.545166 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:01.556303 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.547818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.27.115.254:45721)
0811 18:50:01.548022 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.548026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.548209 (+ 183us) server_negotiation.cc:407] Connection header received
0811 18:50:01.548326 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.548329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.548382 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.548462 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:01.549343 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.549901 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.550824 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.551050 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.551169 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.552739 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:01.552742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:01.552745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:01.552772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:01.554458 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.554909 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.554912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.554913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.554996 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:01.555330 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:01.555332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:01.555333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:01.555584 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:01.555713 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.556028 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.556145 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:50:01.560456 381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:01.563796 381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:01.580401 937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.573479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46657 (local address 127.0.0.1:46160)
0811 18:50:01.574886 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.574947 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.575140 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.575912 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.575914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.575931 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:01.576095 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.576101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577225 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.577232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.578640 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.578650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.579678 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.579680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.579769 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.579802 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.579893 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.580216 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":234,"thread_start_us":105,"threads_started":1}
I20260811 18:50:01.580703 936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.573642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.27.115.193:46657)
0811 18:50:01.574271 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.574281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.575044 (+ 763us) server_negotiation.cc:407] Connection header received
0811 18:50:01.575303 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.575307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.575492 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.575696 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:01.576213 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577063 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.578774 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.579541 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.579687 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.579839 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.579948 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.580193 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":109,"threads_started":1}
I20260811 18:50:01.581490 939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.573798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:45179 (local address 127.0.0.1:57222)
0811 18:50:01.574882 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.574949 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.575107 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.575731 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.575737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.575761 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:01.575911 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.575919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577435 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.577440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.578999 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.579009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.580453 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.580457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.581147 (+ 690us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.581171 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.581246 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.581330 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":74,"spinlock_wait_cycles":5120,"thread_start_us":100,"threads_started":1}
I20260811 18:50:01.581969 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.574454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:34341 (local address 127.0.0.1:51882)
0811 18:50:01.574955 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.574969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.575108 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.575919 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.575923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.575945 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:01.576098 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.576106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577472 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.577474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.578997 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.579009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.581510 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.581516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.581652 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.581684 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.581755 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.581824 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":215,"spinlock_wait_cycles":11392,"thread_start_us":94,"threads_started":1}
I20260811 18:50:01.581990 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.574204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.27.115.195:45179)
0811 18:50:01.574554 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.574567 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.575039 (+ 472us) server_negotiation.cc:407] Connection header received
0811 18:50:01.575326 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.575332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.575559 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.575724 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:01.576040 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577270 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.579143 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.579977 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.580258 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.580386 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.581733 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.581844 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
I20260811 18:50:01.582477 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.574611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.27.115.194:34341)
0811 18:50:01.575396 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.575407 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.575425 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:50:01.575489 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.575494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.575651 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.575791 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:01.576235 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.577339 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.580231 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.581369 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.581512 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.581637 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.581964 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.582330 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":115,"threads_started":1}
I20260811 18:50:01.584108 532 tablet_service.cc:1467] Processing CreateTablet for tablet aaa2b0446aea4530902ad3cedd083600 (DEFAULT_TABLE table=test-table [id=ad78e8a9b61d4d7081d201a572653238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:01.585302 532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaa2b0446aea4530902ad3cedd083600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:01.584957 836 tablet_service.cc:1467] Processing CreateTablet for tablet aaa2b0446aea4530902ad3cedd083600 (DEFAULT_TABLE table=test-table [id=ad78e8a9b61d4d7081d201a572653238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:01.584956 685 tablet_service.cc:1467] Processing CreateTablet for tablet aaa2b0446aea4530902ad3cedd083600 (DEFAULT_TABLE table=test-table [id=ad78e8a9b61d4d7081d201a572653238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:01.586068 685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaa2b0446aea4530902ad3cedd083600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:01.586071 836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaa2b0446aea4530902ad3cedd083600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:01.592496 942 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: Bootstrap starting.
I20260811 18:50:01.593619 943 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Bootstrap starting.
I20260811 18:50:01.596288 942 tablet_bootstrap.cc:654] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:01.596288 943 tablet_bootstrap.cc:654] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:01.597349 942 log.cc:824] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:01.597637 943 log.cc:824] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:01.598649 944 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Bootstrap starting.
I20260811 18:50:01.599917 943 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: No bootstrap required, opened a new log
I20260811 18:50:01.600087 942 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: No bootstrap required, opened a new log
I20260811 18:50:01.600191 943 ts_tablet_manager.cc:1397] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:50:01.600291 942 ts_tablet_manager.cc:1397] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 18:50:01.601294 944 tablet_bootstrap.cc:654] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:01.602818 944 log.cc:824] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:01.604089 943 raft_consensus.cc:348] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.604383 943 raft_consensus.cc:374] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:01.604437 943 raft_consensus.cc:729] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4999c14466544bb18813700132247503, State: Initialized, Role: FOLLOWER
I20260811 18:50:01.604925 943 consensus_queue.cc:260] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.605414 944 tablet_bootstrap.cc:492] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: No bootstrap required, opened a new log
I20260811 18:50:01.605360 942 raft_consensus.cc:348] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.605643 944 ts_tablet_manager.cc:1397] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260811 18:50:01.605636 942 raft_consensus.cc:374] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:01.605710 942 raft_consensus.cc:729] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25642519c5834e90a44eb9d3424b6c71, State: Initialized, Role: FOLLOWER
I20260811 18:50:01.606086 943 ts_tablet_manager.cc:1428] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 18:50:01.606192 942 consensus_queue.cc:260] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.606344 758 heartbeater.cc:503] Master 127.27.115.254:45721 was elected leader, sending a full tablet report...
I20260811 18:50:01.607158 942 ts_tablet_manager.cc:1428] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260811 18:50:01.607542 612 heartbeater.cc:503] Master 127.27.115.254:45721 was elected leader, sending a full tablet report...
I20260811 18:50:01.610388 944 raft_consensus.cc:348] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.610707 944 raft_consensus.cc:374] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:01.610765 944 raft_consensus.cc:729] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5dd7d874a8404ebc6d0259e0e7755c, State: Initialized, Role: FOLLOWER
I20260811 18:50:01.611234 944 consensus_queue.cc:260] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.611974 915 heartbeater.cc:503] Master 127.27.115.254:45721 was elected leader, sending a full tablet report...
I20260811 18:50:01.612349 944 ts_tablet_manager.cc:1428] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:50:01.658605 950 raft_consensus.cc:482] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:01.658833 950 raft_consensus.cc:504] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.660005 950 leader_election.cc:288] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25642519c5834e90a44eb9d3424b6c71 (127.27.115.193:46657), 4999c14466544bb18813700132247503 (127.27.115.194:34341)
I20260811 18:50:01.663996 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.660359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:34341 (local address 127.27.115.195:48803)
0811 18:50:01.660470 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.660488 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.660595 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.660928 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.660932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.660952 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:01.661098 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.661104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.661918 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.661922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.662815 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.662825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.663534 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.663538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.663660 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.663679 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.663770 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.663849 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1,"spinlock_wait_cycles":31616}
I20260811 18:50:01.663995 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.660564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:48803 (local address 127.27.115.194:34341)
0811 18:50:01.660677 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.660681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.660706 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:50:01.660765 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.660767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.660807 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.660910 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:01.661259 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.661796 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.662960 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.663395 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.663534 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.663596 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.663791 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.663867 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:50:01.664223 936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.660452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:37037 (local address 127.27.115.193:46657)
0811 18:50:01.660603 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.660609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.660964 (+ 355us) server_negotiation.cc:407] Connection header received
0811 18:50:01.661154 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.661157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.661211 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.661298 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:01.661565 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.662068 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.663217 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.663657 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.663793 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.663850 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.664030 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.664107 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:50:01.664242 953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.660359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46657 (local address 127.27.115.195:37037)
0811 18:50:01.660868 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.660879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:01.661022 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:01.661309 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:01.661313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:01.661330 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:01.661453 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.661458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.662267 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.662271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.663073 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:01.663082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.663787 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.663791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:01.663896 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.663925 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:50:01.664013 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:50:01.664082 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":127,"spinlock_wait_cycles":384,"thread_start_us":158,"threads_started":1}
I20260811 18:50:01.664875 708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaa2b0446aea4530902ad3cedd083600" candidate_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4999c14466544bb18813700132247503" is_pre_election: true
I20260811 18:50:01.664992 557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaa2b0446aea4530902ad3cedd083600" candidate_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25642519c5834e90a44eb9d3424b6c71" is_pre_election: true
I20260811 18:50:01.665225 708 raft_consensus.cc:2454] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5dd7d874a8404ebc6d0259e0e7755c in term 0.
I20260811 18:50:01.665316 557 raft_consensus.cc:2454] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5dd7d874a8404ebc6d0259e0e7755c in term 0.
I20260811 18:50:01.665791 786 leader_election.cc:302] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25642519c5834e90a44eb9d3424b6c71, 4999c14466544bb18813700132247503, 9e5dd7d874a8404ebc6d0259e0e7755c; no voters:
I20260811 18:50:01.666070 950 raft_consensus.cc:2781] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:01.666200 950 raft_consensus.cc:482] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:01.666260 950 raft_consensus.cc:3037] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:01.667430 950 raft_consensus.cc:504] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.667853 950 leader_election.cc:288] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [CANDIDATE]: Term 1 election: Requested vote from peers 25642519c5834e90a44eb9d3424b6c71 (127.27.115.193:46657), 4999c14466544bb18813700132247503 (127.27.115.194:34341)
I20260811 18:50:01.668309 557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaa2b0446aea4530902ad3cedd083600" candidate_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25642519c5834e90a44eb9d3424b6c71"
I20260811 18:50:01.668371 708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaa2b0446aea4530902ad3cedd083600" candidate_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4999c14466544bb18813700132247503"
I20260811 18:50:01.668483 557 raft_consensus.cc:3037] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:01.668566 708 raft_consensus.cc:3037] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:01.669839 557 raft_consensus.cc:2454] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5dd7d874a8404ebc6d0259e0e7755c in term 1.
I20260811 18:50:01.669852 708 raft_consensus.cc:2454] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5dd7d874a8404ebc6d0259e0e7755c in term 1.
I20260811 18:50:01.670372 786 leader_election.cc:302] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4999c14466544bb18813700132247503, 9e5dd7d874a8404ebc6d0259e0e7755c; no voters:
I20260811 18:50:01.670619 950 raft_consensus.cc:2781] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:01.670949 950 raft_consensus.cc:686] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 1 LEADER]: Becoming Leader. State: Replica: 9e5dd7d874a8404ebc6d0259e0e7755c, State: Running, Role: LEADER
I20260811 18:50:01.671402 950 consensus_queue.cc:237] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } }
I20260811 18:50:01.676968 382 catalog_manager.cc:5495] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5dd7d874a8404ebc6d0259e0e7755c (127.27.115.195). New cstate: current_term: 1 leader_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5dd7d874a8404ebc6d0259e0e7755c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:01.705387 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.701465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.27.115.195:45179)
0811 18:50:01.701605 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:01.701609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:01.701623 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:01.701707 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:01.701709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:01.701781 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:01.701876 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:01.702148 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.702655 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.703600 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:01.703835 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:01.703928 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:01.704323 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:01.704857 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:01.704963 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:01.705100 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:50:01.705167 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:50:01.712193 708 raft_consensus.cc:1261] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 1 FOLLOWER]: Refusing update from remote peer 9e5dd7d874a8404ebc6d0259e0e7755c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:01.712235 557 raft_consensus.cc:1261] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 1 FOLLOWER]: Refusing update from remote peer 9e5dd7d874a8404ebc6d0259e0e7755c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:01.713037 950 consensus_queue.cc:1035] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4999c14466544bb18813700132247503" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:01.713304 954 consensus_queue.cc:1035] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25642519c5834e90a44eb9d3424b6c71" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:01.723304 959 mvcc.cc:205] Tried to move back new op lower bound from 7317398330204274688 to 7317398330049732608. Current Snapshot: MvccSnapshot[applied={T|T < 7317398330204274688}]
I20260811 18:50:01.733700 381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:01.733884 381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:01.737084 381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 043735d0419a4bb48f81a5e32abe1631: Sending DeleteTablet for 3 replicas of tablet aaa2b0446aea4530902ad3cedd083600
I20260811 18:50:01.738209 685 tablet_service.cc:1514] Processing DeleteTablet for tablet aaa2b0446aea4530902ad3cedd083600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:01 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:51882
I20260811 18:50:01.738191 532 tablet_service.cc:1514] Processing DeleteTablet for tablet aaa2b0446aea4530902ad3cedd083600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:01 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46160
I20260811 18:50:01.738998 836 tablet_service.cc:1514] Processing DeleteTablet for tablet aaa2b0446aea4530902ad3cedd083600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:01 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:57222
I20260811 18:50:01.739166 968 tablet_replica.cc:330] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71: stopping tablet replica
I20260811 18:50:01.739563 968 raft_consensus.cc:2229] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:01.740089 967 tablet_replica.cc:330] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: stopping tablet replica
I20260811 18:50:01.740377 968 raft_consensus.cc:2258] T aaa2b0446aea4530902ad3cedd083600 P 25642519c5834e90a44eb9d3424b6c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:01.740473 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 457
I20260811 18:50:01.740660 967 raft_consensus.cc:2229] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:01.745538 967 raft_consensus.cc:2258] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:01.746335 969 tablet_replica.cc:330] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: stopping tablet replica
I20260811 18:50:01.746852 969 raft_consensus.cc:2229] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:01.747437 969 raft_consensus.cc:2258] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:01.749562 969 ts_tablet_manager.cc:1905] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:01.749572 967 ts_tablet_manager.cc:1905] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:01.752887 969 ts_tablet_manager.cc:1918] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:01.753146 969 log.cc:1196] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/aaa2b0446aea4530902ad3cedd083600
I20260811 18:50:01.753597 969 ts_tablet_manager.cc:1939] T aaa2b0446aea4530902ad3cedd083600 P 9e5dd7d874a8404ebc6d0259e0e7755c: Deleting consensus metadata
I20260811 18:50:01.753674 967 ts_tablet_manager.cc:1918] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:01.753819 967 log.cc:1196] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/aaa2b0446aea4530902ad3cedd083600
I20260811 18:50:01.754202 967 ts_tablet_manager.cc:1939] T aaa2b0446aea4530902ad3cedd083600 P 4999c14466544bb18813700132247503: Deleting consensus metadata
I20260811 18:50:01.755514 359 catalog_manager.cc:4841] TS 9e5dd7d874a8404ebc6d0259e0e7755c (127.27.115.195:45179): tablet aaa2b0446aea4530902ad3cedd083600 (table test-table [id=ad78e8a9b61d4d7081d201a572653238]) successfully deleted
I20260811 18:50:01.755741 363 catalog_manager.cc:4841] TS 4999c14466544bb18813700132247503 (127.27.115.194:34341): tablet aaa2b0446aea4530902ad3cedd083600 (table test-table [id=ad78e8a9b61d4d7081d201a572653238]) successfully deleted
W20260811 18:50:01.756338 362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:46657 (error 108)
I20260811 18:50:01.756664 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 617
I20260811 18:50:01.762310 939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:01.761747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46657 (local address 127.0.0.1:46172)
0811 18:50:01.762017 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:01.762180 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:46657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260811 18:50:01.762727 362 catalog_manager.cc:4584] TS 25642519c5834e90a44eb9d3424b6c71 (127.27.115.193:46657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aaa2b0446aea4530902ad3cedd083600: Network error: Client connection negotiation failed: client connection to 127.27.115.193:46657: connect: Connection refused (error 111)
I20260811 18:50:01.768734 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 764
I20260811 18:50:01.779421 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 342
2026-08-11T18:50:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2634 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T18:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:01Z Disabled control of system clock
I20260811 18:50:01.824324 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:34617
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:34617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:50:01.966507 976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:01.966881 976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:01.966923 976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:01.971939 976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:01.972050 976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:01.972069 976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:01.972092 976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:01.972110 976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:01.977705 976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:34617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:01.979079 976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:01.981000 976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:01.991961 983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:01.992125 985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:01.992034 982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:01.992496 976 server_base.cc:1034] running on GCE node
I20260811 18:50:01.993502 976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:01.994954 976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:01.996205 976 hybrid_clock.cc:648] HybridClock initialized: now 1786474201996170 us; error 53 us; skew 500 ppm
I20260811 18:50:01.998775 976 webserver.cc:460] Webserver started at http://127.27.115.254:33015/ using document root <none> and password file <none>
I20260811 18:50:01.999513 976 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:01.999629 976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:01.999903 976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:02.002578 976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "1ac5452e4c144226ba9c61bc5dfe5d67"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "30a6b264e8dee9dc7d4fd2ff2be5ebe8"
server_key_iv: "2369cabcae050035d94259cf1e1ea185"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.003355 976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "1ac5452e4c144226ba9c61bc5dfe5d67"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "30a6b264e8dee9dc7d4fd2ff2be5ebe8"
server_key_iv: "2369cabcae050035d94259cf1e1ea185"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.007980 976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:02.011168 991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.012849 976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:02.013049 976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "1ac5452e4c144226ba9c61bc5dfe5d67"
format_stamp: "Formatted at 2026-08-11 18:50:01 on dist-test-slave-9jjr"
server_key: "30a6b264e8dee9dc7d4fd2ff2be5ebe8"
server_key_iv: "2369cabcae050035d94259cf1e1ea185"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.013234 976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:02.018899 976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:02.019744 976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:02.019976 976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:02.028432 976 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:34617
I20260811 18:50:02.028443 1051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:34617 every 8 connection(s)
I20260811 18:50:02.029577 976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:02.033517 1052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:02.033915 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 976
I20260811 18:50:02.034066 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:02.034421 28111 external_mini_cluster.cc:1428] Setting key 1a8c984ec2f4c3f65765f8d501cfc1c2
I20260811 18:50:02.040545 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.036053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52346 (local address 127.27.115.254:34617)
0811 18:50:02.036751 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.036768 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.036805 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:50:02.037491 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.037508 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.037873 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.038301 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.039133 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.039138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.039150 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.039161 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.039217 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.039235 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.039358 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.039489 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.039788 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":171,"threads_started":1}
I20260811 18:50:02.040601 1052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: Bootstrap starting.
I20260811 18:50:02.043227 1052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:02.044101 1052 log.cc:824] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:02.046718 1052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: No bootstrap required, opened a new log
I20260811 18:50:02.050840 1052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } }
I20260811 18:50:02.051178 1052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:02.051225 1052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac5452e4c144226ba9c61bc5dfe5d67, State: Initialized, Role: FOLLOWER
I20260811 18:50:02.051925 1052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } }
I20260811 18:50:02.052304 1052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:02.052441 1052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:02.052618 1052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:02.053946 1052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } }
I20260811 18:50:02.054409 1052 leader_election.cc:302] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ac5452e4c144226ba9c61bc5dfe5d67; no voters:
I20260811 18:50:02.054795 1052 leader_election.cc:288] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:02.055186 1057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:02.055510 1057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [term 1 LEADER]: Becoming Leader. State: Replica: 1ac5452e4c144226ba9c61bc5dfe5d67, State: Running, Role: LEADER
I20260811 18:50:02.055891 1057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } }
I20260811 18:50:02.056684 1052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:02.057819 1061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ac5452e4c144226ba9c61bc5dfe5d67. Latest consensus state: current_term: 1 leader_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } } }
I20260811 18:50:02.057983 1061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:02.058324 1059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5452e4c144226ba9c61bc5dfe5d67" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 34617 } } }
I20260811 18:50:02.058415 1059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:02.058476 1069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:02.062435 1069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:02.069447 1069 catalog_manager.cc:1347] Generated new cluster ID: 308a0e49569f4bf1a54e467e2f702439
I20260811 18:50:02.069556 1069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:02.081612 1069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:02.082021 1069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:02.088044 1069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: Generated new TSK 0
I20260811 18:50:02.089023 1069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:02.095293 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--builtin_ntp_servers=127.27.115.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:50:02.231477 1084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:02.231813 1084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:02.231858 1084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:02.236415 1084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:02.236498 1084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:02.236559 1084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:02.241658 1084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:02.242990 1084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:02.244580 1084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:02.256335 1094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:02.256412 1096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:02.256345 1092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:02.256865 1084 server_base.cc:1034] running on GCE node
I20260811 18:50:02.257665 1084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:02.258590 1084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:02.266063 1084 hybrid_clock.cc:648] HybridClock initialized: now 1786474202266012 us; error 51 us; skew 500 ppm
I20260811 18:50:02.268520 1084 webserver.cc:460] Webserver started at http://127.27.115.193:35379/ using document root <none> and password file <none>
I20260811 18:50:02.269223 1084 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:02.269296 1084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:02.269531 1084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:02.272034 1084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "820ba93fb4644929827a73533c8ebe16"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "646b78f8a904458e79a90192cef45be4"
server_key_iv: "16b393df262bbb10e9743be5c5709f59"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.272641 1084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "820ba93fb4644929827a73533c8ebe16"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "646b78f8a904458e79a90192cef45be4"
server_key_iv: "16b393df262bbb10e9743be5c5709f59"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.277052 1084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:02.280292 1104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.281831 1084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:02.281991 1084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "820ba93fb4644929827a73533c8ebe16"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "646b78f8a904458e79a90192cef45be4"
server_key_iv: "16b393df262bbb10e9743be5c5709f59"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.282203 1084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:02.287197 1084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:02.287897 1084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:02.288113 1084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:02.288729 1084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:02.289762 1084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:02.289819 1084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.289896 1084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:02.289929 1084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.299714 1084 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:38205
I20260811 18:50:02.299736 1242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:38205 every 8 connection(s)
I20260811 18:50:02.300778 1084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:02.304440 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1084
I20260811 18:50:02.304692 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:02.305061 28111 external_mini_cluster.cc:1428] Setting key 4e4152d2832e6fa453832bb8e4de71ce
I20260811 18:50:02.307650 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--builtin_ntp_servers=127.27.115.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:50:02.308707 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.302933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:39961 (local address 127.27.115.254:34617)
0811 18:50:02.303107 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.303111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.304302 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 18:50:02.306531 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.306535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.306579 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.306660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.308132 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.308136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.308138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.308153 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.308175 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.308184 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.308278 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.308527 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.308576 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:02.310006 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.303293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:34617 (local address 127.27.115.193:39961)
0811 18:50:02.304092 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.304163 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.306290 (+ 2127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.306894 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.306904 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.307032 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.307743 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.307747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.307818 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.307926 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.308291 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.308305 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.308314 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.308526 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.308819 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":129,"threads_started":1}
I20260811 18:50:02.311750 1243 heartbeater.cc:347] Connected to a master server at 127.27.115.254:34617
I20260811 18:50:02.312029 1243 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:02.312548 1243 heartbeater.cc:511] Master 127.27.115.254:34617 requested a full tablet report, sending...
I20260811 18:50:02.314750 1009 ts_manager.cc:194] Registered new tserver with Master: 820ba93fb4644929827a73533c8ebe16 (127.27.115.193:38205)
W20260811 18:50:02.436508 1246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:02.436853 1246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:02.436901 1246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:02.442319 1246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:02.442408 1246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:02.442473 1246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:02.447718 1246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:02.449048 1246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:02.450698 1246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:02.462217 1254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:02.462267 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:02.462674 1246 server_base.cc:1034] running on GCE node
W20260811 18:50:02.462217 1256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:02.463493 1246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:02.464231 1246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:02.465519 1246 hybrid_clock.cc:648] HybridClock initialized: now 1786474202465476 us; error 66 us; skew 500 ppm
I20260811 18:50:02.467931 1246 webserver.cc:460] Webserver started at http://127.27.115.194:36641/ using document root <none> and password file <none>
I20260811 18:50:02.468598 1246 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:02.468679 1246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:02.468924 1246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:02.471364 1246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "acae3395a0ab452f914a290e018facfa"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "17d28edba68d34eecef376a5b4b8a0e9"
server_key_iv: "e5253ba92a75505d3fbab8aaf59906f7"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.472043 1246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "acae3395a0ab452f914a290e018facfa"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "17d28edba68d34eecef376a5b4b8a0e9"
server_key_iv: "e5253ba92a75505d3fbab8aaf59906f7"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.477211 1246 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:50:02.480754 1263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.482384 1246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:50:02.482565 1246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "acae3395a0ab452f914a290e018facfa"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "17d28edba68d34eecef376a5b4b8a0e9"
server_key_iv: "e5253ba92a75505d3fbab8aaf59906f7"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.482753 1246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:02.488304 1246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:02.489007 1246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:02.489223 1246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:02.489846 1246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:02.490922 1246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:02.490998 1246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.491072 1246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:02.491117 1246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.501003 1246 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:39231
I20260811 18:50:02.501062 1397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:39231 every 8 connection(s)
I20260811 18:50:02.502050 1246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:02.507365 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1246
I20260811 18:50:02.507546 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:02.507893 28111 external_mini_cluster.cc:1428] Setting key 3df8a4f18ca71ec4e4d95c8f9e928ac3
I20260811 18:50:02.509224 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.504219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:43875 (local address 127.27.115.254:34617)
0811 18:50:02.504375 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.504378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.505572 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 18:50:02.507056 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.507060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.507157 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.507260 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.508752 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.508755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.508757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.508767 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.508788 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.508796 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.508883 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.509076 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.509116 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:50:02.510416 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--builtin_ntp_servers=127.27.115.212:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:50:02.510515 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.504642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:34617 (local address 127.27.115.194:43875)
0811 18:50:02.505392 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.505438 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.506800 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.507479 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.507491 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.507628 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.508264 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.508266 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.508328 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.508424 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.508904 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.508914 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.508920 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.509119 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.509386 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":132,"threads_started":1}
I20260811 18:50:02.511822 1398 heartbeater.cc:347] Connected to a master server at 127.27.115.254:34617
I20260811 18:50:02.512331 1398 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:02.513126 1398 heartbeater.cc:511] Master 127.27.115.254:34617 requested a full tablet report, sending...
I20260811 18:50:02.514601 1009 ts_manager.cc:194] Registered new tserver with Master: acae3395a0ab452f914a290e018facfa (127.27.115.194:39231)
W20260811 18:50:02.635484 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:02.635818 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:02.635866 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:02.640210 1401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:02.640290 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:02.640352 1401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:02.645569 1401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:34617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:02.646901 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:02.648442 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:02.659487 1407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:02.659487 1410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:02.659734 1408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:02.660115 1401 server_base.cc:1034] running on GCE node
I20260811 18:50:02.660630 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:02.661556 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:02.662776 1401 hybrid_clock.cc:648] HybridClock initialized: now 1786474202662728 us; error 62 us; skew 500 ppm
I20260811 18:50:02.665221 1401 webserver.cc:460] Webserver started at http://127.27.115.195:32889/ using document root <none> and password file <none>
I20260811 18:50:02.665923 1401 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:02.665998 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:02.666298 1401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:02.668813 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "19b5b506934245b8be6e0f7904bbab22"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "c1995ad31f115938feefe99aa0ea8961"
server_key_iv: "535b998b693a258e147bf4900121545e"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.669567 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "19b5b506934245b8be6e0f7904bbab22"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "c1995ad31f115938feefe99aa0ea8961"
server_key_iv: "535b998b693a258e147bf4900121545e"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.673936 1401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:02.676918 1416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.678789 1401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:02.678925 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "19b5b506934245b8be6e0f7904bbab22"
format_stamp: "Formatted at 2026-08-11 18:50:02 on dist-test-slave-9jjr"
server_key: "c1995ad31f115938feefe99aa0ea8961"
server_key_iv: "535b998b693a258e147bf4900121545e"
server_key_version: "encryptionkey@0"
I20260811 18:50:02.679083 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:02.684415 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:02.685081 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:02.685292 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:02.685897 1401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:02.687040 1401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:02.687121 1401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.687182 1401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:02.687237 1401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:02.696621 1401 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:38809
I20260811 18:50:02.696647 1554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:38809 every 8 connection(s)
I20260811 18:50:02.697597 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:02.700042 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1401
I20260811 18:50:02.700183 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:02.700515 28111 external_mini_cluster.cc:1428] Setting key ebb370f9353b7312d4c5c3b08ac0a34b
I20260811 18:50:02.704779 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.699675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:43343 (local address 127.27.115.254:34617)
0811 18:50:02.699847 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.699851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.701308 (+ 1457us) server_negotiation.cc:407] Connection header received
0811 18:50:02.703013 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.703017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.703077 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.703155 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.704275 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.704279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.704282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.704302 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.704322 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.704330 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.704425 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.704583 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.704643 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:50:02.705699 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.700191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:34617 (local address 127.27.115.195:43343)
0811 18:50:02.701160 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.701206 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.702799 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.703333 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.703341 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.703436 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.703995 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.703997 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.704038 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.704115 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.704422 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.704432 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.704439 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.704633 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.704885 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":169,"threads_started":1}
I20260811 18:50:02.706898 1555 heartbeater.cc:347] Connected to a master server at 127.27.115.254:34617
I20260811 18:50:02.707129 1555 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:02.707535 1555 heartbeater.cc:511] Master 127.27.115.254:34617 requested a full tablet report, sending...
I20260811 18:50:02.708632 1009 ts_manager.cc:194] Registered new tserver with Master: 19b5b506934245b8be6e0f7904bbab22 (127.27.115.195:38809)
I20260811 18:50:02.715332 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:02.719688 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.717543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.27.115.254:34617)
0811 18:50:02.717727 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.717734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.718010 (+ 276us) server_negotiation.cc:407] Connection header received
0811 18:50:02.718188 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.718192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.718243 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.718377 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.719175 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.719179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.719181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.719195 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.719219 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.719228 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.719325 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.719398 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.719443 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:02.723635 1009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:02.727020 1009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:02.741150 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.737227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:38809 (local address 127.0.0.1:35504)
0811 18:50:02.738203 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.738270 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.738387 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.739183 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.739190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.739287 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.739980 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.739983 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.740007 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.740093 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.740590 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.740607 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.740617 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.740795 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.740909 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":128,"threads_started":1}
I20260811 18:50:02.741406 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.737734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46516 (local address 127.27.115.194:39231)
0811 18:50:02.738103 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.738119 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.738486 (+ 367us) server_negotiation.cc:407] Connection header received
0811 18:50:02.738705 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.738711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.738950 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.739088 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.740665 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.740671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.740680 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.740691 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.740729 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.740803 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.740904 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.741144 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.741227 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260811 18:50:02.741632 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.738072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38205 (local address 127.0.0.1:41370)
0811 18:50:02.739088 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.739103 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.739253 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.740231 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.740234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.740250 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.740779 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.740782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.740793 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.740829 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.741379 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.741382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.741385 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.741459 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.741500 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":133,"threads_started":1}
I20260811 18:50:02.742049 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.737312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:39231 (local address 127.0.0.1:46516)
0811 18:50:02.738376 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.738390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.738563 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.739696 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.739698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.739715 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.740455 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.740458 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.740469 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.740499 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.741012 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.741015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.741018 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.741859 (+ 841us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.741901 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":137,"threads_started":1}
I20260811 18:50:02.742256 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.737338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.27.115.195:38809)
0811 18:50:02.738682 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.738696 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.738722 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:50:02.738796 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.738804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.739001 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.739179 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.740332 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.740339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.740354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.740368 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.740422 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.740462 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.740606 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.740827 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.741901 (+ 1074us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1230,"thread_start_us":138,"threads_started":1}
I20260811 18:50:02.742282 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.738437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.27.115.193:38205)
0811 18:50:02.738919 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.738930 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.739188 (+ 258us) server_negotiation.cc:407] Connection header received
0811 18:50:02.739385 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.739394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.739584 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.739722 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.741034 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.741041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.741053 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.741071 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.741122 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.741171 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.741291 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.741863 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.742078 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":127,"threads_started":1}
I20260811 18:50:02.746307 1322 tablet_service.cc:1467] Processing CreateTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 (DEFAULT_TABLE table=test-table [id=abbcb16b8f9c408bb9831313d8578e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:02.746299 1159 tablet_service.cc:1467] Processing CreateTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 (DEFAULT_TABLE table=test-table [id=abbcb16b8f9c408bb9831313d8578e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:02.746302 1479 tablet_service.cc:1467] Processing CreateTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 (DEFAULT_TABLE table=test-table [id=abbcb16b8f9c408bb9831313d8578e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:02.747879 1322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2183e64ce0dd4b35b9218b710d2c99f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:02.747895 1479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2183e64ce0dd4b35b9218b710d2c99f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:02.747895 1159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2183e64ce0dd4b35b9218b710d2c99f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:02.756943 1577 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Bootstrap starting.
I20260811 18:50:02.756501 1575 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Bootstrap starting.
I20260811 18:50:02.757499 1576 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Bootstrap starting.
I20260811 18:50:02.759651 1575 tablet_bootstrap.cc:654] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:02.760179 1577 tablet_bootstrap.cc:654] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:02.760483 1576 tablet_bootstrap.cc:654] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:02.761098 1575 log.cc:824] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:02.761132 1577 log.cc:824] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:02.761857 1576 log.cc:824] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:02.763482 1575 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: No bootstrap required, opened a new log
I20260811 18:50:02.763568 1577 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: No bootstrap required, opened a new log
I20260811 18:50:02.763737 1575 ts_tablet_manager.cc:1397] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:50:02.763782 1577 ts_tablet_manager.cc:1397] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:50:02.764072 1576 tablet_bootstrap.cc:492] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: No bootstrap required, opened a new log
I20260811 18:50:02.764256 1576 ts_tablet_manager.cc:1397] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:50:02.767513 1575 raft_consensus.cc:348] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.767786 1575 raft_consensus.cc:374] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:02.767840 1575 raft_consensus.cc:729] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acae3395a0ab452f914a290e018facfa, State: Initialized, Role: FOLLOWER
I20260811 18:50:02.768352 1575 consensus_queue.cc:260] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.768914 1576 raft_consensus.cc:348] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.769212 1576 raft_consensus.cc:374] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:02.769272 1576 raft_consensus.cc:729] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b5b506934245b8be6e0f7904bbab22, State: Initialized, Role: FOLLOWER
I20260811 18:50:02.769537 1575 ts_tablet_manager.cc:1428] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 18:50:02.769383 1577 raft_consensus.cc:348] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.769625 1577 raft_consensus.cc:374] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:02.769685 1577 raft_consensus.cc:729] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820ba93fb4644929827a73533c8ebe16, State: Initialized, Role: FOLLOWER
I20260811 18:50:02.769730 1576 consensus_queue.cc:260] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.769945 1398 heartbeater.cc:503] Master 127.27.115.254:34617 was elected leader, sending a full tablet report...
I20260811 18:50:02.770164 1577 consensus_queue.cc:260] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.770499 1555 heartbeater.cc:503] Master 127.27.115.254:34617 was elected leader, sending a full tablet report...
I20260811 18:50:02.770814 1576 ts_tablet_manager.cc:1428] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 18:50:02.771090 1577 ts_tablet_manager.cc:1428] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 18:50:02.771507 1243 heartbeater.cc:503] Master 127.27.115.254:34617 was elected leader, sending a full tablet report...
W20260811 18:50:02.802198 1244 tablet.cc:2369] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:02.851781 1584 raft_consensus.cc:482] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:02.852021 1584 raft_consensus.cc:504] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.853547 1584 leader_election.cc:288] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acae3395a0ab452f914a290e018facfa (127.27.115.194:39231), 820ba93fb4644929827a73533c8ebe16 (127.27.115.193:38205)
I20260811 18:50:02.855976 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.853864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:58007 (local address 127.27.115.194:39231)
0811 18:50:02.854057 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.854063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.854190 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:50:02.854396 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.854400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.854445 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.854544 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.855433 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.855437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.855441 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.855455 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.855481 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.855494 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.855608 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.855796 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.855843 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:50:02.855983 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.853883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:39231 (local address 127.27.115.195:58007)
0811 18:50:02.854070 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.854087 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.854267 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.854637 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.854640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.854659 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.855232 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.855235 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.855249 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.855279 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.855634 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.855642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.855649 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.855798 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.855842 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 18:50:02.856400 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.854450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:35891 (local address 127.27.115.193:38205)
0811 18:50:02.854649 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.854654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.854798 (+ 144us) server_negotiation.cc:407] Connection header received
0811 18:50:02.854931 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.854935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.854976 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.855062 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.855884 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.855887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.855890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.855902 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.855923 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.855933 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.855996 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.856205 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.856253 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:50:02.856734 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.854275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38205 (local address 127.27.115.195:35891)
0811 18:50:02.854689 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.854705 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:02.854821 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:02.855129 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:02.855135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:02.855158 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:02.855670 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:02.855672 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:02.855690 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:02.855724 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:02.856072 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:02.856077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:02.856081 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:02.856576 (+ 495us) client_negotiation.cc:240] Negotiation successful
0811 18:50:02.856614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":145,"threads_started":1}
I20260811 18:50:02.856755 1345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2183e64ce0dd4b35b9218b710d2c99f0" candidate_uuid: "19b5b506934245b8be6e0f7904bbab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acae3395a0ab452f914a290e018facfa" is_pre_election: true
I20260811 18:50:02.857096 1345 raft_consensus.cc:2454] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b5b506934245b8be6e0f7904bbab22 in term 0.
I20260811 18:50:02.857455 1188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2183e64ce0dd4b35b9218b710d2c99f0" candidate_uuid: "19b5b506934245b8be6e0f7904bbab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820ba93fb4644929827a73533c8ebe16" is_pre_election: true
I20260811 18:50:02.857681 1421 leader_election.cc:302] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b5b506934245b8be6e0f7904bbab22, acae3395a0ab452f914a290e018facfa; no voters:
I20260811 18:50:02.857765 1188 raft_consensus.cc:2454] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19b5b506934245b8be6e0f7904bbab22 in term 0.
I20260811 18:50:02.858314 1584 raft_consensus.cc:2781] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:02.858430 1584 raft_consensus.cc:482] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:02.858495 1584 raft_consensus.cc:3037] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:02.859711 1584 raft_consensus.cc:504] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.860143 1584 leader_election.cc:288] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [CANDIDATE]: Term 1 election: Requested vote from peers acae3395a0ab452f914a290e018facfa (127.27.115.194:39231), 820ba93fb4644929827a73533c8ebe16 (127.27.115.193:38205)
I20260811 18:50:02.860599 1188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2183e64ce0dd4b35b9218b710d2c99f0" candidate_uuid: "19b5b506934245b8be6e0f7904bbab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820ba93fb4644929827a73533c8ebe16"
I20260811 18:50:02.860679 1345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2183e64ce0dd4b35b9218b710d2c99f0" candidate_uuid: "19b5b506934245b8be6e0f7904bbab22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acae3395a0ab452f914a290e018facfa"
I20260811 18:50:02.860731 1188 raft_consensus.cc:3037] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:02.860875 1345 raft_consensus.cc:3037] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:02.862242 1188 raft_consensus.cc:2454] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b5b506934245b8be6e0f7904bbab22 in term 1.
I20260811 18:50:02.862564 1421 leader_election.cc:302] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b5b506934245b8be6e0f7904bbab22, 820ba93fb4644929827a73533c8ebe16; no voters:
I20260811 18:50:02.862684 1345 raft_consensus.cc:2454] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19b5b506934245b8be6e0f7904bbab22 in term 1.
I20260811 18:50:02.862759 1584 raft_consensus.cc:2781] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:02.863047 1584 raft_consensus.cc:686] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 1 LEADER]: Becoming Leader. State: Replica: 19b5b506934245b8be6e0f7904bbab22, State: Running, Role: LEADER
I20260811 18:50:02.863437 1584 consensus_queue.cc:237] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } }
I20260811 18:50:02.867370 1008 catalog_manager.cc:5495] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19b5b506934245b8be6e0f7904bbab22 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "19b5b506934245b8be6e0f7904bbab22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b5b506934245b8be6e0f7904bbab22" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 38809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:02.899788 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.898014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.27.115.195:38809)
0811 18:50:02.898252 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:02.898257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:02.898274 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:02.898378 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:02.898381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:02.898452 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:02.898555 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:02.899342 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:02.899350 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:02.899353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:02.899364 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:02.899383 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:02.899392 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:02.899496 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:02.899590 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:50:02.899633 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260811 18:50:02.906775 1345 raft_consensus.cc:1261] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 1 FOLLOWER]: Refusing update from remote peer 19b5b506934245b8be6e0f7904bbab22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:02.906776 1188 raft_consensus.cc:1261] T 2183e64ce0dd4b35b9218b710d2c99f0 P 820ba93fb4644929827a73533c8ebe16 [term 1 FOLLOWER]: Refusing update from remote peer 19b5b506934245b8be6e0f7904bbab22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:02.907814 1588 consensus_queue.cc:1035] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acae3395a0ab452f914a290e018facfa" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 39231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:02.908066 1584 consensus_queue.cc:1035] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "820ba93fb4644929827a73533c8ebe16" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:02.916971 1593 mvcc.cc:205] Tried to move back new op lower bound from 7317398335096254464 to 7317398334931558400. Current Snapshot: MvccSnapshot[applied={T|T < 7317398335096254464}]
I20260811 18:50:02.926502 1008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:02.926752 1008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:02.929840 1008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ac5452e4c144226ba9c61bc5dfe5d67: Sending DeleteTablet for 3 replicas of tablet 2183e64ce0dd4b35b9218b710d2c99f0
I20260811 18:50:02.930992 1479 tablet_service.cc:1514] Processing DeleteTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:02 UTC) from {username='slave'} at 127.0.0.1:35504
I20260811 18:50:02.931068 1322 tablet_service.cc:1514] Processing DeleteTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:02 UTC) from {username='slave'} at 127.0.0.1:46516
I20260811 18:50:02.931061 1159 tablet_service.cc:1514] Processing DeleteTablet for tablet 2183e64ce0dd4b35b9218b710d2c99f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:02 UTC) from {username='slave'} at 127.0.0.1:41370
I20260811 18:50:02.931685 1603 tablet_replica.cc:330] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: stopping tablet replica
I20260811 18:50:02.931888 1605 tablet_replica.cc:330] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: stopping tablet replica
I20260811 18:50:02.932120 1603 raft_consensus.cc:2229] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:02.932216 1605 raft_consensus.cc:2229] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:02.932580 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1084
I20260811 18:50:02.932626 1603 raft_consensus.cc:2258] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:02.932905 1605 raft_consensus.cc:2258] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:02.942446 1605 ts_tablet_manager.cc:1905] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:02.943783 1603 ts_tablet_manager.cc:1905] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:02.946614 1605 ts_tablet_manager.cc:1918] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:02.946765 1605 log.cc:1196] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/2183e64ce0dd4b35b9218b710d2c99f0
I20260811 18:50:02.947012 1603 ts_tablet_manager.cc:1918] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:02.947160 1605 ts_tablet_manager.cc:1939] T 2183e64ce0dd4b35b9218b710d2c99f0 P acae3395a0ab452f914a290e018facfa: Deleting consensus metadata
I20260811 18:50:02.947281 1603 log.cc:1196] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/2183e64ce0dd4b35b9218b710d2c99f0
I20260811 18:50:02.947703 1603 ts_tablet_manager.cc:1939] T 2183e64ce0dd4b35b9218b710d2c99f0 P 19b5b506934245b8be6e0f7904bbab22: Deleting consensus metadata
I20260811 18:50:02.949034 994 catalog_manager.cc:4841] TS 19b5b506934245b8be6e0f7904bbab22 (127.27.115.195:38809): tablet 2183e64ce0dd4b35b9218b710d2c99f0 (table test-table [id=abbcb16b8f9c408bb9831313d8578e9d]) successfully deleted
W20260811 18:50:02.949411 995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:38205 (error 108)
I20260811 18:50:02.949679 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1246
I20260811 18:50:02.949930 995 catalog_manager.cc:4841] TS acae3395a0ab452f914a290e018facfa (127.27.115.194:39231): tablet 2183e64ce0dd4b35b9218b710d2c99f0 (table test-table [id=abbcb16b8f9c408bb9831313d8578e9d]) successfully deleted
I20260811 18:50:02.956727 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:02.956379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38205 (local address 127.0.0.1:41386)
0811 18:50:02.956538 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:02.956635 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 18:50:02.957069 995 catalog_manager.cc:4584] TS 820ba93fb4644929827a73533c8ebe16 (127.27.115.193:38205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2183e64ce0dd4b35b9218b710d2c99f0: Network error: Client connection negotiation failed: client connection to 127.27.115.193:38205: connect: Connection refused (error 111)
I20260811 18:50:02.970110 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1401
I20260811 18:50:02.980297 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 976
2026-08-11T18:50:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1197 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 18:50:03.010488 28111 mini_oidc.cc:152] Starting JWKS server
I20260811 18:50:03.012809 28111 webserver.cc:460] Webserver started at https://127.0.0.1:38195/ using document root <none> and password file <none>
I20260811 18:50:03.013057 28111 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:50:03.013417 28111 webserver.cc:460] Webserver started at http://127.0.0.1:45711/ using document root <none> and password file <none>
2026-08-11T18:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:03Z Disabled control of system clock
I20260811 18:50:03.027873 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44279
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:44279
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38195/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/testchainca.cert with env {}
W20260811 18:50:03.154366 1615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:03.154702 1615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:03.154747 1615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:03.159145 1615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:03.159237 1615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:03.159253 1615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:03.159271 1615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:03.159286 1615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:50:03.159303 1615 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:50:03.159354 1615 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38195/jwks.json
I20260811 18:50:03.166527 1615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:44279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38195/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:03.167810 1615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:03.168956 1615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:03.178484 1625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.178462 1621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.178469 1622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:03.178828 1615 server_base.cc:1034] running on GCE node
I20260811 18:50:03.179469 1615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:03.180747 1615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:03.181968 1615 hybrid_clock.cc:648] HybridClock initialized: now 1786474203181950 us; error 43 us; skew 500 ppm
I20260811 18:50:03.188594 1615 webserver.cc:460] Webserver started at http://127.27.115.254:33795/ using document root <none> and password file <none>
I20260811 18:50:03.189294 1615 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:03.189402 1615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:03.189656 1615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:03.192056 1615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "85694bafd2d24b699408ea41af877143"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "84965f200d632aa47b3a7025e44d618f"
server_key_iv: "974c984a2a60ce8735a6fad6f0e7a73c"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.192742 1615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "85694bafd2d24b699408ea41af877143"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "84965f200d632aa47b3a7025e44d618f"
server_key_iv: "974c984a2a60ce8735a6fad6f0e7a73c"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.197043 1615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 18:50:03.200053 1632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:03.201386 1615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:03.201550 1615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "85694bafd2d24b699408ea41af877143"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "84965f200d632aa47b3a7025e44d618f"
server_key_iv: "974c984a2a60ce8735a6fad6f0e7a73c"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.201666 1615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:03.224009 1615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:03.224824 1615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:03.225042 1615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:03.233532 1615 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:44279
I20260811 18:50:03.233561 1695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:44279 every 8 connection(s)
I20260811 18:50:03.234647 1615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:03.237128 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1615
I20260811 18:50:03.237370 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:03.237792 28111 external_mini_cluster.cc:1428] Setting key aebc750a2749008e51105a0fce674ba5
I20260811 18:50:03.238423 1696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:03.246965 1696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: Bootstrap starting.
I20260811 18:50:03.249554 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:03.240048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.27.115.254:44279)
0811 18:50:03.240895 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:03.240913 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:03.240954 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:50:03.241906 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:03.241931 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:03.242497 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:03.242830 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:03.243344 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.244367 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.245720 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.246418 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.246565 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.247685 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:03.247690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:03.247705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:03.248121 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:03.248187 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:03.248209 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:03.248336 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:03.248467 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:50:03.248762 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":183,"threads_started":1}
I20260811 18:50:03.249825 1696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:03.251077 1696 log.cc:824] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:03.254443 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:50:03.254719 1696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: No bootstrap required, opened a new log
I20260811 18:50:03.260315 1696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } }
I20260811 18:50:03.260731 1696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:03.260814 1696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85694bafd2d24b699408ea41af877143, State: Initialized, Role: FOLLOWER
I20260811 18:50:03.261409 1696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } }
I20260811 18:50:03.261603 1696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:03.261705 1696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:03.261837 1696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:03.263334 1696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } }
I20260811 18:50:03.263836 1696 leader_election.cc:302] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85694bafd2d24b699408ea41af877143; no voters:
I20260811 18:50:03.264245 1696 leader_election.cc:288] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:03.264412 1703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:03.264722 1703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [term 1 LEADER]: Becoming Leader. State: Replica: 85694bafd2d24b699408ea41af877143, State: Running, Role: LEADER
I20260811 18:50:03.265112 1703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } }
I20260811 18:50:03.265839 1696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:03.267779 1705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85694bafd2d24b699408ea41af877143. Latest consensus state: current_term: 1 leader_uuid: "85694bafd2d24b699408ea41af877143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } } }
I20260811 18:50:03.267990 1705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:03.268193 1704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85694bafd2d24b699408ea41af877143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85694bafd2d24b699408ea41af877143" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44279 } } }
I20260811 18:50:03.268350 1704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143 [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:03.272624 1718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:03.272783 1718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:03.272913 1716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:03.273754 1716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:03.281443 1716 catalog_manager.cc:1347] Generated new cluster ID: a468f584ba4f403b90ca83998c2d2cd3
I20260811 18:50:03.281634 1716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:03.295207 1716 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:03.296340 1716 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:03.307726 1716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85694bafd2d24b699408ea41af877143: Generated new TSK 0
I20260811 18:50:03.308625 1716 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:50:03.348205 1728 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 18:50:03.348398 1728 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:50:03.349077 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:03.321439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.27.115.254:44279)
0811 18:50:03.321627 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:03.321632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:03.321649 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:03.321914 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:03.321917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:03.321984 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:03.322091 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:50:03.322408 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.322971 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.323905 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.324156 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.324270 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.324679 (+ 409us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:50:03.348466 (+ 23787us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:50:03.348897 (+ 431us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 18:50:03.349171 1699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 18:50:03.350703 1727 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:03.321131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44279 (local address 127.0.0.1:38594)
0811 18:50:03.321548 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:03.321567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:03.321721 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:03.322119 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:03.322150 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:03.322160 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:50:03.322290 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:03.322297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.323112 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.323116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:03.323768 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:03.323777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.324282 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.324286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:03.324389 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.324440 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:50:03.348819 (+ 24379us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:50:03.349112 (+ 293us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.115.254:44279: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":150,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.115.254:44279: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 18:50:03.351585 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1615
2026-08-11T18:50:03Z chronyd exiting
I20260811 18:50:03.393131 28111 test_util.cc:175] -----------------------------------------------
I20260811 18:50:03.393260 28111 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474173680671-28111-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (386 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 18:50:03.394595 28111 mini_oidc.cc:152] Starting JWKS server
I20260811 18:50:03.395565 28111 webserver.cc:460] Webserver started at https://127.0.0.1:39501/ using document root <none> and password file <none>
I20260811 18:50:03.395656 28111 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:50:03.395978 28111 webserver.cc:460] Webserver started at http://127.0.0.1:45425/ using document root <none> and password file <none>
2026-08-11T18:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:03Z Disabled control of system clock
I20260811 18:50:03.407594 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:46309
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:46309
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39501/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/testchainca.cert with env {}
W20260811 18:50:03.547652 1734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:03.548017 1734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:03.548062 1734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:03.552525 1734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:03.552615 1734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:03.552631 1734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:03.552649 1734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:03.552665 1734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:50:03.552682 1734 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:50:03.552697 1734 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39501/jwks.json
I20260811 18:50:03.558775 1734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:46309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:46309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39501/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:03.560097 1734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:03.561383 1734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:03.571085 1743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.571112 1741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.571210 1740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:03.571497 1734 server_base.cc:1034] running on GCE node
I20260811 18:50:03.572373 1734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:03.573750 1734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:03.575008 1734 hybrid_clock.cc:648] HybridClock initialized: now 1786474203574986 us; error 39 us; skew 500 ppm
I20260811 18:50:03.577339 1734 webserver.cc:460] Webserver started at http://127.27.115.254:39657/ using document root <none> and password file <none>
I20260811 18:50:03.577983 1734 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:03.578051 1734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:03.578322 1734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:03.580655 1734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "68f4c3c7b9114b8d901154b4146fb697"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "372e261edf92658fc2cc586b9974c1fb"
server_key_iv: "5a5711d8b800842839f5187dfdb61ff5"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.581249 1734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "68f4c3c7b9114b8d901154b4146fb697"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "372e261edf92658fc2cc586b9974c1fb"
server_key_iv: "5a5711d8b800842839f5187dfdb61ff5"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.585466 1734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:03.589345 1751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:03.591125 1734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:50:03.591277 1734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "68f4c3c7b9114b8d901154b4146fb697"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "372e261edf92658fc2cc586b9974c1fb"
server_key_iv: "5a5711d8b800842839f5187dfdb61ff5"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.591470 1734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:03.607671 1734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:03.608495 1734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:03.608723 1734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:03.617043 1734 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:46309
I20260811 18:50:03.617072 1817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:46309 every 8 connection(s)
I20260811 18:50:03.618158 1734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:03.622061 1818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:03.627151 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1734
I20260811 18:50:03.627290 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:03.627628 28111 external_mini_cluster.cc:1428] Setting key 1d040c34f5b84fa5e8e67241b35eebd1
I20260811 18:50:03.629185 1818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: Bootstrap starting.
I20260811 18:50:03.632365 1818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:03.633371 1818 log.cc:824] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:03.635910 1818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: No bootstrap required, opened a new log
I20260811 18:50:03.639709 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:03.629288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.27.115.254:46309)
0811 18:50:03.630154 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:03.630178 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:03.630225 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:50:03.631195 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:03.631221 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:03.631774 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:03.632178 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:03.632718 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.634105 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.635154 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.635863 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.636041 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.637035 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:03.637042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:03.637059 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:03.637594 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:03.637672 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:03.637699 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:03.637811 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:03.638004 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 18:50:03.638366 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":140,"threads_started":1}
I20260811 18:50:03.641050 1818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } }
I20260811 18:50:03.641461 1818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:03.641526 1818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f4c3c7b9114b8d901154b4146fb697, State: Initialized, Role: FOLLOWER
I20260811 18:50:03.642038 1818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } }
I20260811 18:50:03.642270 1818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:03.642354 1818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:03.642494 1818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:03.643965 1818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } }
I20260811 18:50:03.644395 1818 leader_election.cc:302] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68f4c3c7b9114b8d901154b4146fb697; no voters:
I20260811 18:50:03.644907 1818 leader_election.cc:288] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:03.645295 1825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:03.645625 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:50:03.645665 1825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [term 1 LEADER]: Becoming Leader. State: Replica: 68f4c3c7b9114b8d901154b4146fb697, State: Running, Role: LEADER
I20260811 18:50:03.646282 1825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } }
I20260811 18:50:03.646591 1818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:03.648597 1826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68f4c3c7b9114b8d901154b4146fb697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } } }
I20260811 18:50:03.648789 1826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:03.649196 1827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f4c3c7b9114b8d901154b4146fb697. Latest consensus state: current_term: 1 leader_uuid: "68f4c3c7b9114b8d901154b4146fb697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f4c3c7b9114b8d901154b4146fb697" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46309 } } }
I20260811 18:50:03.649307 1827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697 [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:03.652909 1844 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:03.653151 1844 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:03.653302 1842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:03.654310 1842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:03.662662 1842 catalog_manager.cc:1347] Generated new cluster ID: 31a6d5d3ef6d4ac1848646b313496a89
I20260811 18:50:03.662779 1842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:03.673959 1842 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:03.675509 1842 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:03.683992 1842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68f4c3c7b9114b8d901154b4146fb697: Generated new TSK 0
I20260811 18:50:03.685316 1842 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:50:03.720544 1854 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 18:50:03.720711 1854 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:50:03.721691 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:03.703682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.27.115.254:46309)
0811 18:50:03.703871 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:03.703878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:03.704111 (+ 233us) server_negotiation.cc:407] Connection header received
0811 18:50:03.704317 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:03.704320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:03.704379 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:03.704497 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:50:03.704813 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.705391 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.706771 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.707060 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.707170 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.707614 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:50:03.721142 (+ 13528us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39501/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:50:03.721478 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42280: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39501/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 18:50:03.721799 1822 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42280: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39501/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 18:50:03.722247 1853 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:03.703570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:46309 (local address 127.0.0.1:42280)
0811 18:50:03.703959 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:03.703976 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:03.704126 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:03.704534 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:03.704538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:03.704547 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:50:03.704700 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:03.704707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.705512 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.705516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:03.706579 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:03.706592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:03.707211 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:03.707215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:03.707325 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:03.707359 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:50:03.721642 (+ 14283us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39501/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:50:03.721962 (+ 320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.115.254:46309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39501/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":143,"threads_started":1}
I20260811 18:50:03.723197 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1734
2026-08-11T18:50:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (372 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 18:50:03.767387 28111 mini_oidc.cc:152] Starting JWKS server
I20260811 18:50:03.768347 28111 webserver.cc:460] Webserver started at https://127.0.0.1:42769/ using document root <none> and password file <none>
I20260811 18:50:03.768445 28111 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:50:03.768674 28111 webserver.cc:460] Webserver started at http://127.0.0.1:41097/ using document root <none> and password file <none>
2026-08-11T18:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:03Z Disabled control of system clock
I20260811 18:50:03.781589 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:46381
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:46381
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42769/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:50:03.920742 1860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:03.921103 1860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:03.921146 1860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:03.926011 1860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:03.926152 1860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:03.926170 1860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:03.926190 1860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:03.926262 1860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:50:03.926308 1860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:50:03.926335 1860 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42769/jwks.json
I20260811 18:50:03.932014 1860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:46381
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:46381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42769/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:03.933424 1860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:03.935398 1860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:03.947551 1869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.947551 1866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:03.947646 1867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:03.948203 1860 server_base.cc:1034] running on GCE node
I20260811 18:50:03.949293 1860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:03.950996 1860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:03.952289 1860 hybrid_clock.cc:648] HybridClock initialized: now 1786474203952253 us; error 76 us; skew 500 ppm
I20260811 18:50:03.955080 1860 webserver.cc:460] Webserver started at http://127.27.115.254:40045/ using document root <none> and password file <none>
I20260811 18:50:03.955873 1860 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:03.955996 1860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:03.956279 1860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:03.958922 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "6c4adf31126c471d99f998e4fc9d10c0"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "4d04a8bd2a44efdee53a9520b337f0b8"
server_key_iv: "68e08df392e6687129333610d5faf12f"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.959679 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "6c4adf31126c471d99f998e4fc9d10c0"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "4d04a8bd2a44efdee53a9520b337f0b8"
server_key_iv: "68e08df392e6687129333610d5faf12f"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.964775 1860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:50:03.968519 1876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:03.970844 1860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:03.971066 1860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "6c4adf31126c471d99f998e4fc9d10c0"
format_stamp: "Formatted at 2026-08-11 18:50:03 on dist-test-slave-9jjr"
server_key: "4d04a8bd2a44efdee53a9520b337f0b8"
server_key_iv: "68e08df392e6687129333610d5faf12f"
server_key_version: "encryptionkey@0"
I20260811 18:50:03.971244 1860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:03.991051 1860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:03.992394 1860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:03.992729 1860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:04.001663 1860 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:46381
I20260811 18:50:04.001761 1938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:46381 every 8 connection(s)
I20260811 18:50:04.002951 1860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:04.006798 1939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:04.011484 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1860
I20260811 18:50:04.011649 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:04.011952 28111 external_mini_cluster.cc:1428] Setting key 672e8297006ec5f4cf10bf0a991dda92
I20260811 18:50:04.014825 1939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0: Bootstrap starting.
I20260811 18:50:04.017820 1939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:04.018853 1939 log.cc:824] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:04.021293 1939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0: No bootstrap required, opened a new log
I20260811 18:50:04.024266 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:04.013672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.27.115.254:46381)
0811 18:50:04.014593 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:04.014614 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:04.014660 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:50:04.015481 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:04.015504 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:04.016047 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:04.016459 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:04.016997 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.018180 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.019229 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.019766 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.020253 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.021780 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:04.021787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:04.021803 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:04.022349 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:04.022431 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:04.022456 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:04.022594 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:04.022724 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:50:04.023077 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":167,"threads_started":1}
I20260811 18:50:04.025274 1939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } }
I20260811 18:50:04.025628 1939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:04.025693 1939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4adf31126c471d99f998e4fc9d10c0, State: Initialized, Role: FOLLOWER
I20260811 18:50:04.026292 1939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } }
I20260811 18:50:04.026453 1939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:04.026541 1939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:04.026692 1939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:04.028196 1939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } }
I20260811 18:50:04.028645 1939 leader_election.cc:302] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c4adf31126c471d99f998e4fc9d10c0; no voters:
I20260811 18:50:04.028784 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:50:04.029102 1939 leader_election.cc:288] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:04.029238 1947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:04.029436 1947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [term 1 LEADER]: Becoming Leader. State: Replica: 6c4adf31126c471d99f998e4fc9d10c0, State: Running, Role: LEADER
I20260811 18:50:04.030045 1947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } }
I20260811 18:50:04.030853 1939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:04.032439 1948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c4adf31126c471d99f998e4fc9d10c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } } }
I20260811 18:50:04.032627 1948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:04.033038 1949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c4adf31126c471d99f998e4fc9d10c0. Latest consensus state: current_term: 1 leader_uuid: "6c4adf31126c471d99f998e4fc9d10c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4adf31126c471d99f998e4fc9d10c0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 46381 } } }
I20260811 18:50:04.033159 1949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:04.033211 1962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:04.037151 1962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:04.045058 1962 catalog_manager.cc:1347] Generated new cluster ID: f4de7c59ef174f0d80f560f1eadb75d1
I20260811 18:50:04.045171 1962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:04.060746 1962 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:04.062353 1962 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:04.078485 1962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c4adf31126c471d99f998e4fc9d10c0: Generated new TSK 0
I20260811 18:50:04.079603 1962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:04.098277 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:04.091326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.27.115.254:46381)
0811 18:50:04.091529 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:04.091534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:04.091762 (+ 228us) server_negotiation.cc:407] Connection header received
0811 18:50:04.091985 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:04.091988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:04.092051 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:04.092166 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:50:04.092505 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.093151 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.094704 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.095040 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.095358 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.095890 (+ 532us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:50:04.097764 (+ 1874us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42769/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:50:04.098069 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42769/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 18:50:04.098369 1944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42769/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 18:50:04.100281 1979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:04.091199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:46381 (local address 127.0.0.1:47200)
0811 18:50:04.091631 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:04.091650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:04.091809 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:04.092203 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:04.092207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:04.092216 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:50:04.092356 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:04.092364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.093307 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.093312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:04.094440 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:04.094455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.095387 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.095392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:04.095517 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.095562 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:50:04.097983 (+ 2421us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42769/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:50:04.100036 (+ 2053us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.115.254:46381: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42769/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":1739,"thread_start_us":191,"threads_started":1}
I20260811 18:50:04.101086 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1860
E20260811 18:50:04.112766 1981 webserver.cc:533] Webserver: sslize failed:
E20260811 18:50:04.112922 1981 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T18:50:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (381 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T18:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:04Z Disabled control of system clock
I20260811 18:50:04.162925 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:40511
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:33487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:40511
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/testchainca.cert with env {}
W20260811 18:50:04.292953 1985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:04.293316 1985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:04.293362 1985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:04.298382 1985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:04.298491 1985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:04.298508 1985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:04.298529 1985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:04.298547 1985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:50:04.298566 1985 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:50:04.298583 1985 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 18:50:04.305534 1985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:40511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:40511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:04.306964 1985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:04.308200 1985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:04.319046 1991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:04.319029 1992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:04.319029 1995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:04.320253 1985 server_base.cc:1034] running on GCE node
I20260811 18:50:04.321151 1985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:04.322840 1985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:04.324082 1985 hybrid_clock.cc:648] HybridClock initialized: now 1786474204324031 us; error 74 us; skew 500 ppm
I20260811 18:50:04.326960 1985 webserver.cc:460] Webserver started at http://127.27.115.254:37175/ using document root <none> and password file <none>
I20260811 18:50:04.327782 1985 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:04.327880 1985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:04.328184 1985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:04.331410 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "16b121ba441e43e78b60419f2a19cd64"
format_stamp: "Formatted at 2026-08-11 18:50:04 on dist-test-slave-9jjr"
server_key: "92fd80268282b1eeb39c5c892b6d2950"
server_key_iv: "345b69b049989389dfdcc7c4ce892389"
server_key_version: "encryptionkey@0"
I20260811 18:50:04.332295 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "16b121ba441e43e78b60419f2a19cd64"
format_stamp: "Formatted at 2026-08-11 18:50:04 on dist-test-slave-9jjr"
server_key: "92fd80268282b1eeb39c5c892b6d2950"
server_key_iv: "345b69b049989389dfdcc7c4ce892389"
server_key_version: "encryptionkey@0"
I20260811 18:50:04.337416 1985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 18:50:04.341423 2001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:04.343832 1985 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:04.344097 1985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "16b121ba441e43e78b60419f2a19cd64"
format_stamp: "Formatted at 2026-08-11 18:50:04 on dist-test-slave-9jjr"
server_key: "92fd80268282b1eeb39c5c892b6d2950"
server_key_iv: "345b69b049989389dfdcc7c4ce892389"
server_key_version: "encryptionkey@0"
I20260811 18:50:04.344290 1985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:04.369823 1985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:04.370772 1985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:04.371017 1985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:04.379415 1985 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:40511
I20260811 18:50:04.379428 2061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:40511 every 8 connection(s)
I20260811 18:50:04.380718 1985 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:04.382704 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 1985
I20260811 18:50:04.382894 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:04.383323 28111 external_mini_cluster.cc:1428] Setting key b8d7aa0ca8a89bc499b676a30147037a
I20260811 18:50:04.385581 2063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:04.392747 2063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64: Bootstrap starting.
I20260811 18:50:04.395651 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:04.385243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.27.115.254:40511)
0811 18:50:04.386293 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:04.386316 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:04.386365 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:50:04.387147 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:04.387170 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:04.387656 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:04.388122 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:04.388699 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.390037 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.391221 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.391790 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.392003 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.393336 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:04.393342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:04.393359 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:04.393930 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:04.394019 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:04.394052 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:04.394224 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:04.394400 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 18:50:04.394794 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":202,"threads_started":1}
I20260811 18:50:04.395864 2063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:04.396705 2063 log.cc:824] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:04.399991 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:50:04.400926 2063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64: No bootstrap required, opened a new log
I20260811 18:50:04.405476 2063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } }
I20260811 18:50:04.405920 2063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:04.406013 2063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b121ba441e43e78b60419f2a19cd64, State: Initialized, Role: FOLLOWER
I20260811 18:50:04.406666 2063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } }
I20260811 18:50:04.406836 2063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:04.406886 2063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:04.406975 2063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:04.408144 2063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } }
I20260811 18:50:04.408540 2063 leader_election.cc:302] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16b121ba441e43e78b60419f2a19cd64; no voters:
I20260811 18:50:04.408900 2063 leader_election.cc:288] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:04.409262 2069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:04.409698 2069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [term 1 LEADER]: Becoming Leader. State: Replica: 16b121ba441e43e78b60419f2a19cd64, State: Running, Role: LEADER
I20260811 18:50:04.410247 2069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } }
I20260811 18:50:04.410737 2063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:04.413015 2072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16b121ba441e43e78b60419f2a19cd64. Latest consensus state: current_term: 1 leader_uuid: "16b121ba441e43e78b60419f2a19cd64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } } }
I20260811 18:50:04.413197 2072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:04.413769 2071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16b121ba441e43e78b60419f2a19cd64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b121ba441e43e78b60419f2a19cd64" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 40511 } } }
I20260811 18:50:04.413964 2071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:04.414505 2081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:04.418582 2081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:04.426935 2081 catalog_manager.cc:1347] Generated new cluster ID: 8224e553edc04c20aefcedc76a982f4a
I20260811 18:50:04.427062 2081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:04.450057 2081 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:04.451927 2081 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:04.460824 2081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16b121ba441e43e78b60419f2a19cd64: Generated new TSK 0
I20260811 18:50:04.461961 2081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:05.355351 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:04.478763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.27.115.254:40511)
0811 18:50:04.478913 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:04.478917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:04.479177 (+ 260us) server_negotiation.cc:407] Connection header received
0811 18:50:04.479317 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:04.479320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:04.479390 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:04.479538 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:50:04.479845 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.480916 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.482247 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.482611 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.482746 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.483439 (+ 693us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:50:05.354256 (+870817us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:50:05.355122 (+ 866us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47194: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 18:50:05.355384 2100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:04.478514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:40511 (local address 127.0.0.1:47194)
0811 18:50:04.478977 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:04.478991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:04.479127 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:04.479571 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:04.479574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:04.479589 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:50:04.479704 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:04.479709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.481218 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.481224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:04.482007 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:04.482020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:04.482874 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:04.482879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:04.483021 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:04.483101 (+ 80us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:50:05.354877 (+871776us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:50:05.355172 (+ 295us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.115.254:40511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":154,"threads_started":1}
W20260811 18:50:05.355489 2066 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47194: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260811 18:50:05.356844 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 1985
2026-08-11T18:50:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1234 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (150 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (177 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:05 dist-test-slave-9jjr krb5kdc[2112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:05 dist-test-slave-9jjr krb5kdc[2112](info): set up 2 sockets
Aug 11 18:50:05 dist-test-slave-9jjr krb5kdc[2112](info): commencing operation
krb5kdc: starting...
W20260811 18:50:06.880988 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:06 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:50:07.035703 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45115
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:07.170931 2127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:07.171308 2127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:07.171409 2127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:07.175781 2127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:07.175894 2127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:07.175935 2127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:07.175993 2127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:07.176051 2127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:07.182410 2127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:07.183745 2127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:07.184932 2127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:07.195762 2135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:07.195762 2137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:07.195789 2134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:07.195873 2127 server_base.cc:1034] running on GCE node
I20260811 18:50:07.196924 2127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:07.198474 2127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:07.201910 2127 hybrid_clock.cc:648] HybridClock initialized: now 1786474207201831 us; error 87 us; skew 500 ppm
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474207, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:07.207124 2127 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:50:07.208583 2127 webserver.cc:460] Webserver started at http://127.27.115.254:45981/ using document root <none> and password file <none>
I20260811 18:50:07.209349 2127 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:07.209436 2127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:07.209731 2127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:07.213398 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "af917f4305b14719a0b397d78a254956"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "c102f3fd3aadddbfd0097d5792cf6e3e"
server_key_iv: "ba10ccc70491a1e1ee0c7017b6683379"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.214217 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "af917f4305b14719a0b397d78a254956"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "c102f3fd3aadddbfd0097d5792cf6e3e"
server_key_iv: "ba10ccc70491a1e1ee0c7017b6683379"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.219278 2127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:50:07.223035 2145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.224788 2127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:07.224973 2127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "af917f4305b14719a0b397d78a254956"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "c102f3fd3aadddbfd0097d5792cf6e3e"
server_key_iv: "ba10ccc70491a1e1ee0c7017b6683379"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.225150 2127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:07.255627 2127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:07.260519 2127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:07.260815 2127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:07.271140 2127 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:45115
I20260811 18:50:07.271173 2201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:45115 every 8 connection(s)
I20260811 18:50:07.272871 2127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:07.275915 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2127
I20260811 18:50:07.276083 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:07.276422 28111 external_mini_cluster.cc:1428] Setting key eb28d9d71087f795fa23577db8e54414
I20260811 18:50:07.277607 2203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:07.286208 2203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956: Bootstrap starting.
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:07.289950 2203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:07.290938 2203 log.cc:824] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:07.294343 2203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956: No bootstrap required, opened a new log
I20260811 18:50:07.295589 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:07.278069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.27.115.254:45115)
0811 18:50:07.278737 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:07.278754 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:07.278796 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:50:07.279698 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:07.279720 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:07.280143 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:07.280583 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:07.281581 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.282830 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.283826 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.284409 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.284624 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:07.288701 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:07.288709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:07.288730 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:07.288774 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:07.291128 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.292046 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.292057 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.292067 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.292470 (+ 403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.292890 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.292895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.292897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.293532 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:07.293752 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:07.294076 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 18:50:07.294472 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":198,"threads_started":1}
I20260811 18:50:07.298368 2203 raft_consensus.cc:348] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } }
I20260811 18:50:07.298832 2203 raft_consensus.cc:374] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:07.298928 2203 raft_consensus.cc:729] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af917f4305b14719a0b397d78a254956, State: Initialized, Role: FOLLOWER
I20260811 18:50:07.299589 2203 consensus_queue.cc:260] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } }
I20260811 18:50:07.299796 2203 raft_consensus.cc:388] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:07.299912 2203 raft_consensus.cc:482] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:07.300067 2203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:07.301450 2203 raft_consensus.cc:504] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } }
I20260811 18:50:07.301921 2203 leader_election.cc:302] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af917f4305b14719a0b397d78a254956; no voters:
I20260811 18:50:07.302443 2203 leader_election.cc:288] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:07.302714 2209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:07.303021 2209 raft_consensus.cc:686] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [term 1 LEADER]: Becoming Leader. State: Replica: af917f4305b14719a0b397d78a254956, State: Running, Role: LEADER
I20260811 18:50:07.303470 2209 consensus_queue.cc:237] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } }
I20260811 18:50:07.304109 2203 sys_catalog.cc:564] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:07.305522 2211 sys_catalog.cc:455] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af917f4305b14719a0b397d78a254956. Latest consensus state: current_term: 1 leader_uuid: "af917f4305b14719a0b397d78a254956" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } } }
I20260811 18:50:07.305706 2211 sys_catalog.cc:458] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:07.306030 2210 sys_catalog.cc:455] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af917f4305b14719a0b397d78a254956" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af917f4305b14719a0b397d78a254956" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45115 } } }
I20260811 18:50:07.306188 2210 sys_catalog.cc:458] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:07.306354 2218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:07.310313 2218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:07.320207 2218 catalog_manager.cc:1347] Generated new cluster ID: 8bbaf21787eb48d1bb97c41a4ec1618d
I20260811 18:50:07.320327 2218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:07.337358 2218 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:07.338539 2218 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:07.345413 2218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af917f4305b14719a0b397d78a254956: Generated new TSK 0
I20260811 18:50:07.346364 2218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:50:07.394954 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:07.542531 2233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:07.542910 2233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:07.542968 2233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:07.548409 2233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:07.548532 2233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:07.548604 2233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:07.555413 2233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:07.556849 2233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:07.558300 2233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:50:07.571221 2233 server_base.cc:1034] running on GCE node
W20260811 18:50:07.571165 2240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:07.571117 2242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:07.571117 2239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:07.572207 2233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:07.573108 2233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:07.574627 2233 hybrid_clock.cc:648] HybridClock initialized: now 1786474207574528 us; error 129 us; skew 500 ppm
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474207, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:07.579252 2233 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:50:07.580447 2233 webserver.cc:460] Webserver started at http://127.27.115.193:38191/ using document root <none> and password file <none>
I20260811 18:50:07.581166 2233 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:07.581247 2233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:07.581511 2233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:07.584206 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "16bf3825f8764887b929b3c0fdd92357"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "466bdc674fe40f052747907f0b9754d4"
server_key_iv: "b228eeb61ccc0ffb4421fa5e3190aba4"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.584975 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "16bf3825f8764887b929b3c0fdd92357"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "466bdc674fe40f052747907f0b9754d4"
server_key_iv: "b228eeb61ccc0ffb4421fa5e3190aba4"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.590098 2233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:50:07.593812 2250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.595522 2233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:50:07.595712 2233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "16bf3825f8764887b929b3c0fdd92357"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "466bdc674fe40f052747907f0b9754d4"
server_key_iv: "b228eeb61ccc0ffb4421fa5e3190aba4"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.595865 2233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:07.626119 2233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:07.630215 2233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:07.630461 2233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:07.631245 2233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:07.632292 2233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:07.632369 2233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.632453 2233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:07.632498 2233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.645463 2233 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:41301
I20260811 18:50:07.645587 2385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:41301 every 8 connection(s)
I20260811 18:50:07.646745 2233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:07.656528 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2233
I20260811 18:50:07.656731 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:07.657034 28111 external_mini_cluster.cc:1428] Setting key 6c41f64d65ce252f0d6dba5521bd7efe
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474207, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:07.667972 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:07.649104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56859 (local address 127.27.115.254:45115)
0811 18:50:07.649532 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:07.649538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:07.650813 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 18:50:07.652894 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:07.652898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:07.652980 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:07.653102 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:07.654943 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.655538 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.657693 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.658044 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.658172 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:07.661975 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:07.661978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:07.661981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:07.662010 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:07.663738 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.665107 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.665116 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.665120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.665309 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.665996 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.666001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.666003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.666416 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:07.666532 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:07.667628 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0811 18:50:07.667806 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260811 18:50:07.669365 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:07.649910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45115 (local address 127.27.115.193:56859)
0811 18:50:07.650616 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:07.650669 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:07.652430 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:07.653507 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:07.653518 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:07.654023 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:07.654725 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:07.654743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.655720 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.655723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:07.657474 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:07.657499 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.658324 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.658329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:07.658605 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:07.659650 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:07.659669 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:07.661671 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:07.663977 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:07.663990 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:07.664029 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:07.664613 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:07.665474 (+ 861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:07.665480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:07.665485 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:07.665757 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:07.666571 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:07.666583 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:07.667059 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 18:50:07.667338 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 18:50:07.667824 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":233,"threads_started":1}
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:07.671154 2386 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45115
I20260811 18:50:07.671693 2386 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:07.672394 2386 heartbeater.cc:511] Master 127.27.115.254:45115 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:50:07.675956 2165 ts_manager.cc:194] Registered new tserver with Master: 16bf3825f8764887b929b3c0fdd92357 (127.27.115.193:41301)
I20260811 18:50:07.677789 2165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:56859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:50:07.698525 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:07.838939 2393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:07.839324 2393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:07.839416 2393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:07.844053 2393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:07.844214 2393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:07.844326 2393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:07.850720 2393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:07.852224 2393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:07.854264 2393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:07.868436 2398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:07.868516 2393 server_base.cc:1034] running on GCE node
W20260811 18:50:07.868439 2401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:07.868439 2399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:07.869421 2393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:07.870409 2393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:07.871655 2393 hybrid_clock.cc:648] HybridClock initialized: now 1786474207871607 us; error 72 us; skew 500 ppm
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474207, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:07.875739 2393 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:50:07.876922 2393 webserver.cc:460] Webserver started at http://127.27.115.194:40681/ using document root <none> and password file <none>
I20260811 18:50:07.877597 2393 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:07.877681 2393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:07.877882 2393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:07.881577 2393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "810b53387b464dbd9b1eadfd0652e960"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "6d449d7f896f46f2ec5b50051f811ab9"
server_key_iv: "f2ba782d23b83442fa83c9b1ae0c01ef"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.882375 2393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "810b53387b464dbd9b1eadfd0652e960"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "6d449d7f896f46f2ec5b50051f811ab9"
server_key_iv: "f2ba782d23b83442fa83c9b1ae0c01ef"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.887609 2393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:50:07.891656 2409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.893945 2393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 18:50:07.894240 2393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "810b53387b464dbd9b1eadfd0652e960"
format_stamp: "Formatted at 2026-08-11 18:50:07 on dist-test-slave-9jjr"
server_key: "6d449d7f896f46f2ec5b50051f811ab9"
server_key_iv: "f2ba782d23b83442fa83c9b1ae0c01ef"
server_key_version: "encryptionkey@0"
I20260811 18:50:07.894454 2393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:07.914960 2393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:07.918999 2393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:07.919212 2393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:07.919847 2393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:07.920861 2393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:07.920918 2393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.920964 2393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:07.920979 2393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:07.933303 2393 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:38403
I20260811 18:50:07.933359 2544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:38403 every 8 connection(s)
I20260811 18:50:07.934643 2393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:07.940487 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2393
I20260811 18:50:07.940618 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:07.941008 28111 external_mini_cluster.cc:1428] Setting key 476eb755a3456cd8c6717a2f35ab3093
Aug 11 18:50:07 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474207, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:50:07.956617 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:07.937325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:59223 (local address 127.27.115.254:45115)
0811 18:50:07.937558 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:07.937565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:07.938909 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 18:50:07.940908 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:07.940914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:07.941000 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:07.941129 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:07.943219 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.944019 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.945627 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.945877 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.946013 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:07.950171 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:07.950181 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:07.950186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:07.950259 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:07.952667 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.953714 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.953721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.953723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.953844 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:07.954584 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:07.954588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:07.954590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:07.954960 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:07.955103 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:07.956242 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0811 18:50:07.956429 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 18:50:07.958061 2547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:07.937941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45115 (local address 127.27.115.194:59223)
0811 18:50:07.938690 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:07.938759 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:07.940506 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:07.941449 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:07.941484 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:07.942093 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:07.942871 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:07.942899 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.944271 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.944275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:07.945447 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:07.945463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:07.946095 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:07.946099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:07.946357 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:07.947263 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:07.947287 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:07.949650 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:07.952955 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:07.952964 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:07.952983 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:07.953524 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:07.954035 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:07.954039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:07.954040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:07.954419 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:07.955198 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:07.955208 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:07.955800 (+ 592us) client_negotiation.cc:769] Sending connection context
0811 18:50:07.956106 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 18:50:07.956757 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":261,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:50:07.960427 2545 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45115
I20260811 18:50:07.960814 2545 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:07.961498 2545 heartbeater.cc:511] Master 127.27.115.254:45115 requested a full tablet report, sending...
I20260811 18:50:07.963187 2165 ts_manager.cc:194] Registered new tserver with Master: 810b53387b464dbd9b1eadfd0652e960 (127.27.115.194:38403)
I20260811 18:50:07.963914 2165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:59223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:50:07.982920 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:08.113698 2550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:08.114035 2550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:08.114084 2550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:08.118959 2550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:08.119060 2550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:08.119130 2550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:08.126909 2550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:45115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:08.128296 2550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:08.129642 2550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:08.141548 2556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:08.141830 2560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:08.141548 2558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:08.142050 2550 server_base.cc:1034] running on GCE node
I20260811 18:50:08.142825 2550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:08.143780 2550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:08.145030 2550 hybrid_clock.cc:648] HybridClock initialized: now 1786474208145013 us; error 42 us; skew 500 ppm
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.148619 2550 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:50:08.149896 2550 webserver.cc:460] Webserver started at http://127.27.115.195:41887/ using document root <none> and password file <none>
I20260811 18:50:08.150645 2550 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:08.150725 2550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:08.150924 2550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:08.153313 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "7d61a2c157b540debfc7f9425f7099fd"
format_stamp: "Formatted at 2026-08-11 18:50:08 on dist-test-slave-9jjr"
server_key: "7b7af1614fc448796a049006ec8d0eee"
server_key_iv: "cf19923736ac4fd1cd08fa9999bd2694"
server_key_version: "encryptionkey@0"
I20260811 18:50:08.153921 2550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "7d61a2c157b540debfc7f9425f7099fd"
format_stamp: "Formatted at 2026-08-11 18:50:08 on dist-test-slave-9jjr"
server_key: "7b7af1614fc448796a049006ec8d0eee"
server_key_iv: "cf19923736ac4fd1cd08fa9999bd2694"
server_key_version: "encryptionkey@0"
I20260811 18:50:08.159493 2550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 18:50:08.162891 2567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:08.164459 2550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:08.164656 2550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "7d61a2c157b540debfc7f9425f7099fd"
format_stamp: "Formatted at 2026-08-11 18:50:08 on dist-test-slave-9jjr"
server_key: "7b7af1614fc448796a049006ec8d0eee"
server_key_iv: "cf19923736ac4fd1cd08fa9999bd2694"
server_key_version: "encryptionkey@0"
I20260811 18:50:08.164798 2550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:08.179762 2550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:08.183887 2550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:08.184136 2550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:08.184811 2550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:08.185752 2550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:08.185801 2550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:08.185873 2550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:08.185910 2550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:08.197108 2550 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:45939
I20260811 18:50:08.197146 2702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:45939 every 8 connection(s)
I20260811 18:50:08.198228 2550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:08.203310 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2550
I20260811 18:50:08.203505 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:08.203815 28111 external_mini_cluster.cc:1428] Setting key 5150db4b65ee6253402eba2cc6a724c4
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:08.222527 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:08.200611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:38721 (local address 127.27.115.254:45115)
0811 18:50:08.200769 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.200773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.201826 (+ 1053us) server_negotiation.cc:407] Connection header received
0811 18:50:08.203507 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.203510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.203582 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.203683 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.205522 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.206381 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.208424 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.208660 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.208768 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:08.218294 (+ 9526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:08.218300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:08.218306 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:08.218346 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:08.220059 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.220726 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.220731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.220733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.220824 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.221223 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.221228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.221230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.221443 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:08.221558 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:08.222189 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 18:50:08.222343 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:50:08.223634 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:08.201015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45115 (local address 127.27.115.195:38721)
0811 18:50:08.201652 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:08.201708 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:08.203254 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:08.203930 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:08.203940 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:08.204498 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:08.205339 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:08.205358 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.206697 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.206702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:08.208212 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:08.208234 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.208836 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.208839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:08.209043 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:08.215941 (+ 6898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:08.215966 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:08.217979 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:08.220236 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:08.220244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:08.220274 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:08.220611 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:08.220921 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:08.220924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:08.220926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:08.221120 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:08.221558 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:08.221568 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:08.221886 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 18:50:08.222106 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 18:50:08.222537 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":190,"threads_started":1}
I20260811 18:50:08.225157 2703 heartbeater.cc:347] Connected to a master server at 127.27.115.254:45115
I20260811 18:50:08.225461 2703 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:08.226091 2703 heartbeater.cc:511] Master 127.27.115.254:45115 requested a full tablet report, sending...
I20260811 18:50:08.227445 2165 ts_manager.cc:194] Registered new tserver with Master: 7d61a2c157b540debfc7f9425f7099fd (127.27.115.195:45939)
I20260811 18:50:08.228104 2165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:38721
I20260811 18:50:08.231407 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/test-admin.keytab.
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.252487 28111 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 18:50:08.256290 2710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:08.254539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.27.115.193:41301)
0811 18:50:08.254994 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.255006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.255037 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:50:08.255104 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.255109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.255327 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.255505 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.256136 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45216: BlockingRecv error: recv got EOF from 127.0.0.1:45216 (error 108)
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":129,"threads_started":1}
W20260811 18:50:08.256633 28111 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.257928 28111 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.259265 28111 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:50:08.272282 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:08.260673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.27.115.193:41301)
0811 18:50:08.260874 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.260880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.260993 (+ 113us) server_negotiation.cc:407] Connection header received
0811 18:50:08.261174 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.261177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.261237 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.261338 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.261973 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.262887 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.263826 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.264464 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.264651 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:08.267811 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:08.267827 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:08.267866 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:08.267904 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:08.269934 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.270633 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.270639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.270641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.270744 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.271124 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.271127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.271128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.271419 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:08.271587 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:08.271891 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 18:50:08.272081 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 18:50:08.276988 2710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:08.275576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.27.115.193:41301)
0811 18:50:08.275703 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.275708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.275811 (+ 103us) server_negotiation.cc:407] Connection header received
0811 18:50:08.275981 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.275985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.276036 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.276121 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.276719 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45238: BlockingRecv error: recv got EOF from 127.0.0.1:45238 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:50:08.277042 28111 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.278493 28111 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.280066 28111 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:50:08.292310 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:08.281365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.27.115.193:41301)
0811 18:50:08.281528 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.281534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.281645 (+ 111us) server_negotiation.cc:407] Connection header received
0811 18:50:08.281773 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.281776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.281820 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.281895 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.282595 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.283150 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.284351 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.284596 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.284694 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:08.287861 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:08.287874 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:08.287876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:08.287916 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:08.289787 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.290623 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.290627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.290629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.290754 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.291117 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.291121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.291123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.291372 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:08.291525 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:08.291906 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 18:50:08.292100 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 18:50:08.296098 2710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:08.294862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.27.115.193:41301)
0811 18:50:08.295056 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.295060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.295207 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:50:08.295368 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.295371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.295419 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.295518 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.295978 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45244: BlockingRecv error: recv got EOF from 127.0.0.1:45244 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 18:50:08.296531 28111 init.cc:244] Error while opening credential cache '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474173680671-28111-0/krb5kdc/krb5cc)
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.297784 28111 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:08.299310 28111 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:50:08.309901 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:08.300662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.27.115.193:41301)
0811 18:50:08.300879 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:08.300884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:08.301090 (+ 206us) server_negotiation.cc:407] Connection header received
0811 18:50:08.301298 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:08.301301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:08.301353 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:08.301454 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:08.302036 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.302665 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.303543 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:08.303801 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:08.303930 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:08.306691 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:08.306694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:08.306697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:08.306729 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:08.308146 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.308619 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.308623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.308624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.308707 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:08.308999 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:08.309003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:08.309004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:08.309189 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:08.309359 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:08.309598 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 18:50:08.309727 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:50:08.311123 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2233
I20260811 18:50:08.322260 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2393
I20260811 18:50:08.331646 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2550
I20260811 18:50:08.343828 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2127
2026-08-11T18:50:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2659 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2724](info): set up 2 sockets
Aug 11 18:50:08 dist-test-slave-9jjr krb5kdc[2724](info): commencing operation
krb5kdc: starting...
W20260811 18:50:09.501648 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:09 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab.
Entry for principal oryx/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab.
I20260811 18:50:09.642848 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37959
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab
--principal=oryx/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:09.772332 2739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:09.772706 2739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:09.772809 2739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:09.777136 2739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:09.777271 2739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:09.777305 2739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:09.777335 2739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:09.777372 2739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:09.783754 2739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab
--principal=oryx/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:09.785204 2739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:09.786573 2739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:09.797396 2745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:09.797549 2739 server_base.cc:1034] running on GCE node
W20260811 18:50:09.797422 2748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:09.797431 2744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:09.799005 2739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:09.800523 2739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:09.801808 2739 hybrid_clock.cc:648] HybridClock initialized: now 1786474209801781 us; error 48 us; skew 500 ppm
Aug 11 18:50:09 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:09.805464 2739 init.cc:377] Logged in from keytab as oryx/127.27.115.254@KRBTEST.COM (short username oryx)
I20260811 18:50:09.806914 2739 webserver.cc:460] Webserver started at http://127.27.115.254:44443/ using document root <none> and password file <none>
I20260811 18:50:09.807801 2739 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:09.807957 2739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:09.808270 2739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:09.810915 2739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "df2278625c904d828175bedfdac5757b"
format_stamp: "Formatted at 2026-08-11 18:50:09 on dist-test-slave-9jjr"
server_key: "10e26a511c447e3355a17b28714961f8"
server_key_iv: "17cdecaaf753e3a2726ce8c35b8213e5"
server_key_version: "encryptionkey@0"
I20260811 18:50:09.811621 2739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "df2278625c904d828175bedfdac5757b"
format_stamp: "Formatted at 2026-08-11 18:50:09 on dist-test-slave-9jjr"
server_key: "10e26a511c447e3355a17b28714961f8"
server_key_iv: "17cdecaaf753e3a2726ce8c35b8213e5"
server_key_version: "encryptionkey@0"
I20260811 18:50:09.816316 2739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:09.819516 2755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:09.821280 2739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:09.821512 2739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "df2278625c904d828175bedfdac5757b"
format_stamp: "Formatted at 2026-08-11 18:50:09 on dist-test-slave-9jjr"
server_key: "10e26a511c447e3355a17b28714961f8"
server_key_iv: "17cdecaaf753e3a2726ce8c35b8213e5"
server_key_version: "encryptionkey@0"
I20260811 18:50:09.821683 2739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:09.850860 2739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:09.854530 2739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:09.854805 2739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:09.864073 2739 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37959
I20260811 18:50:09.864382 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37959 every 8 connection(s)
I20260811 18:50:09.865370 2739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:09.868578 2815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:09.872694 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2739
I20260811 18:50:09.872860 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:09.873167 28111 external_mini_cluster.cc:1428] Setting key 3ac8407b366e54197f8b51025b634bd2
I20260811 18:50:09.875770 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Bootstrap starting.
I20260811 18:50:09.878443 2815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:09.879515 2815 log.cc:824] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:09.882485 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: No bootstrap required, opened a new log
Aug 11 18:50:09 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:09.887621 2815 raft_consensus.cc:348] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } }
I20260811 18:50:09.887974 2815 raft_consensus.cc:374] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:09.888041 2815 raft_consensus.cc:729] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2278625c904d828175bedfdac5757b, State: Initialized, Role: FOLLOWER
I20260811 18:50:09.888687 2815 consensus_queue.cc:260] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } }
I20260811 18:50:09.888880 2815 raft_consensus.cc:388] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:09.888957 2815 raft_consensus.cc:482] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:09.889055 2815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:09.890467 2815 raft_consensus.cc:504] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } }
I20260811 18:50:09.890873 2815 leader_election.cc:302] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df2278625c904d828175bedfdac5757b; no voters:
I20260811 18:50:09.891239 2815 leader_election.cc:288] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:09.891502 2821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:09.891840 2821 raft_consensus.cc:686] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 LEADER]: Becoming Leader. State: Replica: df2278625c904d828175bedfdac5757b, State: Running, Role: LEADER
I20260811 18:50:09.892030 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:09.874890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.27.115.254:37959)
0811 18:50:09.875417 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:09.875444 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:09.875485 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:50:09.876276 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:09.876297 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:09.876678 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:09.877079 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:09.878004 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:09.879224 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:09.880396 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:09.881125 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:09.881741 (+ 616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:09.884587 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:09.884594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:09.884614 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:09.884649 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:09.887590 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:09.888243 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:09.888248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:09.888254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:09.888417 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:09.888750 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:09.888753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:09.888755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:09.889360 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:09.890393 (+ 1033us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:09.890570 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 18:50:09.890906 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":146,"threads_started":1}
I20260811 18:50:09.892604 2821 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } }
I20260811 18:50:09.892745 2815 sys_catalog.cc:564] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:09.896938 2823 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: New leader df2278625c904d828175bedfdac5757b. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } }
I20260811 18:50:09.897048 2823 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:09.896662 2822 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } }
I20260811 18:50:09.897279 2822 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:09.900146 2837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:09.900270 2837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:09.900337 2838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:09.901361 2838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:09.911533 2838 catalog_manager.cc:1347] Generated new cluster ID: 779bc23929264bf68fbbe11102d3abcf
I20260811 18:50:09.911644 2838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:09.924335 2838 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:09.925812 2838 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:09.937131 2838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Generated new TSK 0
I20260811 18:50:09.938259 2838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
I20260811 18:50:09.979980 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:10.109040 2845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:10.109376 2845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:10.109426 2845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:10.113961 2845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:10.114061 2845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:10.114189 2845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:10.120733 2845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:10.122181 2845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:10.123441 2845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:10.135142 2854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:10.135321 2845 server_base.cc:1034] running on GCE node
W20260811 18:50:10.135147 2852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.135147 2851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:10.136045 2845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:10.136893 2845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:10.138144 2845 hybrid_clock.cc:648] HybridClock initialized: now 1786474210138114 us; error 48 us; skew 500 ppm
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:10.141572 2845 init.cc:377] Logged in from keytab as oryx/127.27.115.193@KRBTEST.COM (short username oryx)
I20260811 18:50:10.142768 2845 webserver.cc:460] Webserver started at http://127.27.115.193:37889/ using document root <none> and password file <none>
I20260811 18:50:10.143456 2845 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:10.143563 2845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:10.143831 2845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:10.146368 2845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "78d1d62ac15443d1a945d68c8dad4473"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "f6014cda4a0de0048752ee9030804c88"
server_key_iv: "8333b0f0a163b1d9b6a9dc1085b4a292"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.147131 2845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "78d1d62ac15443d1a945d68c8dad4473"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "f6014cda4a0de0048752ee9030804c88"
server_key_iv: "8333b0f0a163b1d9b6a9dc1085b4a292"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.151878 2845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:10.154891 2862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.156397 2845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:10.156585 2845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "78d1d62ac15443d1a945d68c8dad4473"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "f6014cda4a0de0048752ee9030804c88"
server_key_iv: "8333b0f0a163b1d9b6a9dc1085b4a292"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.156733 2845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:10.172904 2845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:10.176831 2845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:10.177052 2845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:10.177740 2845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:10.178853 2845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:10.178903 2845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.178946 2845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:10.178962 2845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.189097 2845 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:39833
I20260811 18:50:10.189154 2993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:39833 every 8 connection(s)
I20260811 18:50:10.190200 2845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:10.199029 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 2845
I20260811 18:50:10.199218 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:10.199579 28111 external_mini_cluster.cc:1428] Setting key dc2b66f06027ca2ead78c4ba1aaa66a2
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:10.207551 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.192497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:47939 (local address 127.27.115.254:37959)
0811 18:50:10.192656 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:10.192660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:10.193729 (+ 1069us) server_negotiation.cc:407] Connection header received
0811 18:50:10.195508 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:10.195511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:10.195571 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:10.195665 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:10.197009 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.197588 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.199143 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.199531 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.199676 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.202776 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:10.202783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:10.202790 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:10.202832 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:10.204619 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.205409 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.205415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.205420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.205535 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.205935 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.205941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.205945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.206300 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:10.206435 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:10.207086 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 18:50:10.207302 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:50:10.208573 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.192841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:47939)
0811 18:50:10.193512 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:10.193592 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:10.195226 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:10.195869 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:10.195876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:10.196294 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:10.196835 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.196850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.197752 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.197755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.198973 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.198986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.199821 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.199826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.200009 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.200707 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:10.200735 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:10.202559 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:10.204785 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.204796 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.204820 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.205275 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.205644 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.205646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.205647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.205816 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.206453 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:10.206461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:10.206758 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 18:50:10.206963 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 18:50:10.207525 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":162,"threads_started":1}
I20260811 18:50:10.210384 2994 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:10.210711 2994 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:10.211320 2994 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:10.213193 2774 ts_manager.cc:194] Registered new tserver with Master: 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833)
I20260811 18:50:10.214598 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:47939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
I20260811 18:50:10.233963 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:10.361968 3001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:10.362372 3001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:10.362425 3001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:10.366696 3001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:10.366778 3001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:10.366842 3001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:10.372746 3001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:10.373950 3001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:10.375051 3001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:10.385932 3010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.385982 3007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.385982 3006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:10.386399 3001 server_base.cc:1034] running on GCE node
I20260811 18:50:10.387024 3001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:10.387949 3001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:10.389174 3001 hybrid_clock.cc:648] HybridClock initialized: now 1786474210389152 us; error 40 us; skew 500 ppm
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:10.392722 3001 init.cc:377] Logged in from keytab as oryx/127.27.115.194@KRBTEST.COM (short username oryx)
I20260811 18:50:10.393885 3001 webserver.cc:460] Webserver started at http://127.27.115.194:45725/ using document root <none> and password file <none>
I20260811 18:50:10.394615 3001 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:10.394724 3001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:10.394970 3001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:10.397331 3001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "b1bb239449d94043b1a7425c86b760b9"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "0bc9e8b183b99e3913116eaf85e5018c"
server_key_iv: "89f8ce98b9754adbd9cc6d0fa3ad533b"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.398010 3001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "b1bb239449d94043b1a7425c86b760b9"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "0bc9e8b183b99e3913116eaf85e5018c"
server_key_iv: "89f8ce98b9754adbd9cc6d0fa3ad533b"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.402472 3001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:10.405352 3018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.407068 3001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:10.407260 3001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b1bb239449d94043b1a7425c86b760b9"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "0bc9e8b183b99e3913116eaf85e5018c"
server_key_iv: "89f8ce98b9754adbd9cc6d0fa3ad533b"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.407410 3001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:10.432425 3001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:10.436290 3001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:10.436540 3001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:10.437289 3001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:10.438321 3001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:10.438395 3001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.438472 3001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:10.438522 3001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.448998 3001 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:32825
I20260811 18:50:10.449018 3149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:32825 every 8 connection(s)
I20260811 18:50:10.450007 3001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:10.453039 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3001
I20260811 18:50:10.453172 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:10.453500 28111 external_mini_cluster.cc:1428] Setting key 21e3c29ba993b413393b4485afcf2ba6
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:50:10.466066 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.452104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37495 (local address 127.27.115.254:37959)
0811 18:50:10.452260 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:10.452264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:10.453073 (+ 809us) server_negotiation.cc:407] Connection header received
0811 18:50:10.454781 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:10.454784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:10.454843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:10.454958 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:10.456215 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.456751 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.457962 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.458299 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.458427 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.461400 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:10.461407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:10.461413 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:10.461455 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:10.463412 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.464176 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.464180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.464182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.464292 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.464709 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.464715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.464721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.464941 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:10.465063 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:10.465720 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 18:50:10.465887 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:50:10.467192 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.452429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:37495)
0811 18:50:10.452905 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:10.452953 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:10.454548 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:10.455147 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:10.455154 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:10.455534 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:10.456056 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.456070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.456899 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.456903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.457812 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.457823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.458516 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.458520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.458686 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.459374 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:10.459392 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:10.461047 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:10.463620 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.463627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.463646 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.464022 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.464400 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.464403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.464404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.464569 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.465082 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:10.465092 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:10.465429 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 18:50:10.465663 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 18:50:10.466101 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.195@KRBTEST.COM" created.
I20260811 18:50:10.468815 3150 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:10.469128 3150 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:10.469832 3150 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
I20260811 18:50:10.471457 2774 ts_manager.cc:194] Registered new tserver with Master: b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:10.472394 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:37495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
I20260811 18:50:10.487380 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:10.613147 3155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:10.613461 3155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:10.613536 3155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:10.617736 3155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:10.617810 3155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:10.617908 3155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:10.623590 3155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:10.624821 3155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:10.625927 3155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:10.636463 3164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.636466 3162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.636510 3161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:10.636904 3155 server_base.cc:1034] running on GCE node
I20260811 18:50:10.637369 3155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:10.638058 3155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:10.639240 3155 hybrid_clock.cc:648] HybridClock initialized: now 1786474210639205 us; error 66 us; skew 500 ppm
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:10.642431 3155 init.cc:377] Logged in from keytab as oryx/127.27.115.195@KRBTEST.COM (short username oryx)
I20260811 18:50:10.643535 3155 webserver.cc:460] Webserver started at http://127.27.115.195:40179/ using document root <none> and password file <none>
I20260811 18:50:10.644202 3155 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:10.644284 3155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:10.644552 3155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:10.647101 3155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "57a994af048149b3b8de443ab4ab7de4"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "56ea4e73eb0b8bb4f980017625171238"
server_key_iv: "11b3bb9894d76c61ba8ff2677726f805"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.647820 3155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "57a994af048149b3b8de443ab4ab7de4"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "56ea4e73eb0b8bb4f980017625171238"
server_key_iv: "11b3bb9894d76c61ba8ff2677726f805"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.652690 3155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:50:10.657186 3174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.659284 3155 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:10.659513 3155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "57a994af048149b3b8de443ab4ab7de4"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "56ea4e73eb0b8bb4f980017625171238"
server_key_iv: "11b3bb9894d76c61ba8ff2677726f805"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.659770 3155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:10.687899 3155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:10.692183 3155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:10.692482 3155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:10.693256 3155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:10.694283 3155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:10.694360 3155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.694484 3155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:10.694530 3155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.705016 3155 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:35015
I20260811 18:50:10.705067 3307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:35015 every 8 connection(s)
I20260811 18:50:10.706051 3155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:10.707077 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3155
I20260811 18:50:10.707227 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:10.707587 28111 external_mini_cluster.cc:1428] Setting key 7cc06459c121a19ed3aa2b5c0f3d3812
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:10.725009 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.709465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:40227 (local address 127.27.115.254:37959)
0811 18:50:10.709630 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:10.709635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:10.710868 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 18:50:10.712444 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:10.712447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:10.712504 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:10.712602 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:10.714116 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.714730 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.715992 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.716249 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.716346 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.719517 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:10.719522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:10.719524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:10.719557 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:10.722284 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.723145 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.723150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.723151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.723289 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:10.723729 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:10.723732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:10.723734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:10.723959 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:10.724077 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:10.724678 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 18:50:10.724832 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:50:10.726624 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:10.710009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:40227)
0811 18:50:10.710669 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:10.710731 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:10.712182 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:10.712900 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:10.712908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:10.713377 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:10.713932 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.713947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.714871 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.714874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.715851 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:10.715862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:10.716413 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:10.716415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:10.716580 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:10.717332 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:10.717351 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:10.719183 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:10.722464 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.722471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.722493 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.723009 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.723385 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:10.723388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:10.723390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:10.723616 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:10.724081 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:10.724092 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:10.724410 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 18:50:10.724627 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 18:50:10.725032 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":194,"threads_started":1}
I20260811 18:50:10.728058 3308 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:10.728364 3308 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:10.728963 3308 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
I20260811 18:50:10.730263 2774 ts_manager.cc:194] Registered new tserver with Master: 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:10.731055 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:40227
I20260811 18:50:10.736182 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.115.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.253.keytab.
Entry for principal oryx/127.27.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.253.keytab.
I20260811 18:50:10.774657 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:38273
--webserver_interface=127.27.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.253.keytab
--principal=oryx/127.27.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37959,127.27.115.253:38273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:10.916463 3314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:10.916821 3314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:10.916903 3314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:10.921586 3314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:10.921741 3314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:10.921782 3314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:10.921825 3314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:10.921865 3314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:10.928881 3314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37959,127.27.115.253:38273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.253.keytab
--principal=oryx/127.27.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:38273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
W20260811 18:50:10.929227 3314 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.115.254:37959,127.27.115.253:38273'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 18:50:10.930342 3314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:10.931483 3314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:10.941076 3321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.941283 3320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:10.941102 3323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:10.941952 3314 server_base.cc:1034] running on GCE node
I20260811 18:50:10.942991 3314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:10.944926 3314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:10.946204 3314 hybrid_clock.cc:648] HybridClock initialized: now 1786474210946170 us; error 39 us; skew 500 ppm
Aug 11 18:50:10 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:10.950598 3314 init.cc:377] Logged in from keytab as oryx/127.27.115.253@KRBTEST.COM (short username oryx)
I20260811 18:50:10.952168 3314 webserver.cc:460] Webserver started at http://127.27.115.253:40533/ using document root <none> and password file <none>
I20260811 18:50:10.953289 3314 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:10.953419 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:10.953728 3314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:10.957227 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/instance:
uuid: "bead63396fa34de09de801712b80edb2"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "4ea13e6500b7accee1a0dffba0c0aa7d"
server_key_iv: "da1a2a4371188179a1065d7fa3991a01"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.958093 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal/instance:
uuid: "bead63396fa34de09de801712b80edb2"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "4ea13e6500b7accee1a0dffba0c0aa7d"
server_key_iv: "da1a2a4371188179a1065d7fa3991a01"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.964357 3314 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:50:10.968547 3334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:10.970459 3314 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:10.970651 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "bead63396fa34de09de801712b80edb2"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "4ea13e6500b7accee1a0dffba0c0aa7d"
server_key_iv: "da1a2a4371188179a1065d7fa3991a01"
server_key_version: "encryptionkey@0"
I20260811 18:50:10.970822 3314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:10.999969 3314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.004894 3314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:11.005156 3314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.016538 3314 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.253:38273
I20260811 18:50:11.016819 3394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.253:38273 every 8 connection(s)
I20260811 18:50:11.017639 3314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
I20260811 18:50:11.020632 3395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:11.025187 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3314
I20260811 18:50:11.024935 3395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.253@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
I20260811 18:50:11.042364 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.027942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.27.115.254:37959)
0811 18:50:11.028074 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.028079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.028388 (+ 309us) server_negotiation.cc:407] Connection header received
0811 18:50:11.028812 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.028816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.028866 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.028975 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.030319 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.031286 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.033071 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.033353 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.033481 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.037190 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.037194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.037196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.037229 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.039560 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.040317 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.040321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.040323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.040440 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.040833 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.040838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.040840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.041193 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.041325 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.041971 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.042182 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:50:11.043376 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.027145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.27.115.253:38273)
0811 18:50:11.027626 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.027640 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.027681 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:50:11.028572 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.028593 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.028838 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.029026 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.030104 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.031635 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.033138 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.033845 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.034206 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.037750 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.037757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.037772 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.037825 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.040345 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.040897 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.040904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.040905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.040983 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.041259 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.041264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.041266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.041669 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.041842 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.042192 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.042410 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":331,"spinlock_wait_cycles":1408,"thread_start_us":135,"threads_started":1}
I20260811 18:50:11.043377 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.027870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52332)
0811 18:50:11.028230 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.028285 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.028618 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.029117 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.029122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.029661 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.030062 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.030091 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.031470 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.031473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.032914 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.032930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.033551 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.033556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.033749 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.034662 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.034689 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.036944 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.039729 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.039742 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.039761 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.040186 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.040544 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.040551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.040553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.040727 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.041315 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.041322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.041676 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.041914 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.042342 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260811 18:50:11.045935 3395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474210, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.253@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
I20260811 18:50:11.057124 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.046695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.27.115.253:38273)
0811 18:50:11.046800 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.046806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.046845 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:50:11.047030 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.047032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.047073 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.047170 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.047617 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.048217 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.049042 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.050027 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.050241 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.052895 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.052899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.052901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.052931 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.054898 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.055483 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.055488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.055490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.055590 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.055954 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.055958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.055960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.056235 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.056376 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.056691 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.056946 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":190,"server-negotiator.queue_time_us":46}
I20260811 18:50:11.057132 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.046580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40620)
0811 18:50:11.046774 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.046790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.046908 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.047181 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.047184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.047397 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.047499 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.047505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.048344 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.048349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.048903 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.048912 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.050254 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.050258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.050405 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.051118 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.051135 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.052697 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.055079 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.055085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.055088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.055348 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.055717 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.055723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.055725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.055828 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.056360 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.056364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.056478 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.056626 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.056876 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":1}
I20260811 18:50:11.061309 3395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Bootstrap starting.
I20260811 18:50:11.064066 3395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:11.064864 3395 log.cc:824] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:11.067214 3395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: No bootstrap required, opened a new log
I20260811 18:50:11.070559 3395 raft_consensus.cc:348] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } }
I20260811 18:50:11.070828 3395 raft_consensus.cc:374] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:11.070919 3395 raft_consensus.cc:729] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bead63396fa34de09de801712b80edb2, State: Initialized, Role: FOLLOWER
I20260811 18:50:11.071473 3395 consensus_queue.cc:260] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } }
I20260811 18:50:11.072047 3401 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } } }
I20260811 18:50:11.072180 3401 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:11.072834 3395 sys_catalog.cc:564] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:50:11.077888 3412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:11.077998 3412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:11.080415 3314 master_runner.cc:428] Detected that this master bead63396fa34de09de801712b80edb2 is joining an existing cluster
I20260811 18:50:11.080507 3314 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.115.253:38273
I20260811 18:50:11.092056 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.082793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40628)
0811 18:50:11.083305 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.083318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.083441 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.083703 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.083706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.083990 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.084109 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.084115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.085150 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.085154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.086236 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.086250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.086861 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.086949 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.087492 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.087505 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.087891 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.090123 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090157 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.090159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.090450 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.090776 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.090781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.090871 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.091347 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.091350 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.091444 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.091597 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.091812 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":59,"spinlock_wait_cycles":6400,"thread_start_us":130,"threads_started":1}
I20260811 18:50:11.092146 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.082874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.27.115.254:37959)
0811 18:50:11.083014 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.083019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.083351 (+ 332us) server_negotiation.cc:407] Connection header received
0811 18:50:11.083563 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.083567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.083631 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.083687 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.084422 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.085169 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086120 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.086450 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086553 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.088493 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.088497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.088501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.088570 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.090202 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090718 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.090723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.090726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.090823 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.091152 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.091155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.091157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.091410 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.091521 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.091842 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.091981 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:50:11.092072 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.083242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40628 (local address 127.27.115.253:38273)
0811 18:50:11.083340 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.083347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.083382 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:50:11.083554 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.083557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.083600 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.083691 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.084227 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.085018 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086364 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.086732 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086850 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.088044 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.088048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.088051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.088080 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.089976 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090555 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.090561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.090563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.090675 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090995 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.090998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.091000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.091251 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.091361 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.091692 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.091884 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":3200}
I20260811 18:50:11.092901 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.082775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52334)
0811 18:50:11.083255 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.083271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.083447 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.083849 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.083852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.084135 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.084281 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.084291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.085286 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.085291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.085983 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.085992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.086580 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.086584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.086719 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.087837 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.087856 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.088321 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.090328 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.090334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.090595 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.090933 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.090936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.090938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.091040 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.091521 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.091527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.091630 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.092511 (+ 881us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.092701 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":71,"spinlock_wait_cycles":54912,"thread_start_us":127,"threads_started":1}
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.254@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.27.115.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.252@KRBTEST.COM" created.
I20260811 18:50:11.111634 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.100010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.27.115.253:38273)
0811 18:50:11.100161 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.100166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.100180 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:11.100248 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.100251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.100303 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.100385 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.101373 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.102232 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.103779 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.104133 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.104254 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.107480 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.107485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.107488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.107523 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.109302 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.109869 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.109873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.109874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.109971 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.110352 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.110356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.110358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.110610 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.110800 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.111270 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.111435 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:11.112169 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.099164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40638)
0811 18:50:11.099599 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.099667 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.099806 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.100468 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.100477 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.101030 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.101224 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.101236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.102377 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.102384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.103629 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.103645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.104289 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.104296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.104459 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.105198 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.105221 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.107273 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.109439 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.109450 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.109456 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.109756 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.110072 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.110074 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.110075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.110239 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.110805 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.110816 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.110974 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.111821 (+ 847us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.112038 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":120,"threads_started":1}
I20260811 18:50:11.113917 2774 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.115.253:38273: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bead63396fa34de09de801712b80edb2"
member_type: NON_VOTER
last_known_addr {
host: "127.27.115.253"
port: 38273
}
attrs {
promote: true
}
}
dest_uuid: "df2278625c904d828175bedfdac5757b"
cas_config_opid_index: -1
I20260811 18:50:11.114893 2774 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } }
I20260811 18:50:11.116917 3427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bead63396fa34de09de801712b80edb2 (127.27.115.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } }
I20260811 18:50:11.117630 3365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:11.119251 3365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 18:50:11.119495 3427 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.115.253:38273
I20260811 18:50:11.120154 3314 master.cc:537] Master@127.27.115.253:38273 shutting down...
I20260811 18:50:11.120712 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } }
I20260811 18:50:11.120867 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:11.120823 3426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
W20260811 18:50:11.124595 2756 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 18:50:11.125792 2756 consensus_peers.cc:487] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b -> Peer bead63396fa34de09de801712b80edb2 (127.27.115.253:38273): Couldn't send request to peer bead63396fa34de09de801712b80edb2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:50:11.126211 3314 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:11.126607 3314 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:11.126729 3314 tablet_replica.cc:330] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.252.keytab.
Entry for principal oryx/127.27.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.252.keytab.
I20260811 18:50:11.134116 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:33509
--webserver_interface=127.27.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.252.keytab
--principal=oryx/127.27.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:11.142812 3314 master.cc:552] Master@127.27.115.253:38273 shutdown complete.
I20260811 18:50:11.142911 3314 master_runner.cc:305] Clearing existing system tablet
I20260811 18:50:11.143224 3314 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.143318 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.144690 3314 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:11.145766 3437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.146314 3314 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.146504 3314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:11.146551 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "bead63396fa34de09de801712b80edb2"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "4ea13e6500b7accee1a0dffba0c0aa7d"
server_key_iv: "da1a2a4371188179a1065d7fa3991a01"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.149029 3314 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:11.151786 3314 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:50:11.151952 3314 log.cc:1196] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 18:50:11.152398 3314 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Deleting consensus metadata
I20260811 18:50:11.152679 3314 master_runner.cc:315] Copying system tablet from 127.27.115.254:37959
I20260811 18:50:11.154381 3314 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: tablet copy: Beginning tablet copy session from remote peer at address 127.27.115.254:37959
I20260811 18:50:11.164060 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.155287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.27.115.254:37959)
0811 18:50:11.155451 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.155457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.155750 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:50:11.156005 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.156008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.156071 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.156176 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.156715 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.157280 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.158249 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.158489 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.158610 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.159943 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.159947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.159949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.159988 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.162015 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.162678 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.162682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.162683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.162773 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.163095 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.163101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.163105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.163322 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.163441 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.163737 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.163865 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:11.164080 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.155213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52350)
0811 18:50:11.155633 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.155653 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.155865 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.156199 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.156202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.156439 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.156566 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.156573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.157416 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.157420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.158039 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.158049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.158644 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.158648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.158758 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.159378 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.159392 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.159772 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.162250 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.162253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.162254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.162563 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.162878 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.162882 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.162885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.162982 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.163431 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.163434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.163555 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.163678 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.163866 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":126,"threads_started":1}
I20260811 18:50:11.165114 2796 tablet_copy_service.cc:140] P df2278625c904d828175bedfdac5757b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bead63396fa34de09de801712b80edb2 ({username='oryx', principal='oryx/127.27.115.253@KRBTEST.COM'} at 127.0.0.1:52350)
I20260811 18:50:11.165335 2796 tablet_copy_service.cc:161] P df2278625c904d828175bedfdac5757b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bead63396fa34de09de801712b80edb2 at {username='oryx', principal='oryx/127.27.115.253@KRBTEST.COM'} at 127.0.0.1:52350: session id = bead63396fa34de09de801712b80edb2-00000000000000000000000000000000
I20260811 18:50:11.167464 2796 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:50:11.169601 3314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:50:11.172307 3314 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: tablet copy: Starting download of 0 data blocks...
I20260811 18:50:11.172574 3314 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: tablet copy: Starting download of 1 WAL segments...
I20260811 18:50:11.175067 3314 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:50:11.177016 2796 tablet_copy_service.cc:342] P df2278625c904d828175bedfdac5757b: Request end of tablet copy session bead63396fa34de09de801712b80edb2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.115.253@KRBTEST.COM'} at 127.0.0.1:52350
I20260811 18:50:11.177182 2796 tablet_copy_service.cc:433] P df2278625c904d828175bedfdac5757b: ending tablet copy session bead63396fa34de09de801712b80edb2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bead63396fa34de09de801712b80edb2
I20260811 18:50:11.185992 3314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:11.187964 3447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.188153 3450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.188365 3314 server_base.cc:1034] running on GCE node
W20260811 18:50:11.188573 3448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.188892 3314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:11.189316 3314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:11.190541 3314 hybrid_clock.cc:648] HybridClock initialized: now 1786474211190524 us; error 67 us; skew 500 ppm
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:11.192430 3314 init.cc:377] Logged in from keytab as oryx/127.27.115.253@KRBTEST.COM (short username oryx)
I20260811 18:50:11.192976 3314 webserver.cc:460] Webserver started at http://127.27.115.253:42627/ using document root <none> and password file <none>
I20260811 18:50:11.193192 3314 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.193228 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.195003 3314 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:11.196437 3458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.196707 3314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:11.196797 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "bead63396fa34de09de801712b80edb2"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "4ea13e6500b7accee1a0dffba0c0aa7d"
server_key_iv: "da1a2a4371188179a1065d7fa3991a01"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.196880 3314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.213160 3314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.215643 3314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.217445 2994 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
I20260811 18:50:11.220983 3314 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.253:38273
I20260811 18:50:11.221407 3518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.253:38273 every 8 connection(s)
I20260811 18:50:11.221714 3314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
I20260811 18:50:11.223771 3519 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:11.224974 3519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Bootstrap starting.
I20260811 18:50:11.237696 3519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:11.238101 3519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: Bootstrap complete.
I20260811 18:50:11.238863 3519 raft_consensus.cc:348] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } }
I20260811 18:50:11.239045 3519 raft_consensus.cc:729] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bead63396fa34de09de801712b80edb2, State: Initialized, Role: LEARNER
I20260811 18:50:11.239158 3519 consensus_queue.cc:260] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } }
I20260811 18:50:11.239686 3519 sys_catalog.cc:564] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:11.239681 3522 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } }
I20260811 18:50:11.239976 3522 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:50:11.243445 3535 catalog_manager.cc:1259] Loaded cluster ID: 779bc23929264bf68fbbe11102d3abcf
I20260811 18:50:11.243518 3535 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: loading cluster ID for follower catalog manager: success
I20260811 18:50:11.245812 3535 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: acquiring CA information for follower catalog manager: success
I20260811 18:50:11.247632 3535 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:50:11.296319 3433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:11.296725 3433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:11.296809 3433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:11.301543 3433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:11.301649 3433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:11.301672 3433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:11.301689 3433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:11.301705 3433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:11.307744 3433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.252.keytab
--principal=oryx/127.27.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:33509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:11.308996 3433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:11.310171 3433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:11.319553 3540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.319530 3544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.319516 3541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.320148 3433 server_base.cc:1034] running on GCE node
I20260811 18:50:11.320819 3433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:11.322096 3433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:11.323338 3433 hybrid_clock.cc:648] HybridClock initialized: now 1786474211323326 us; error 34 us; skew 500 ppm
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:11.326717 3433 init.cc:377] Logged in from keytab as oryx/127.27.115.252@KRBTEST.COM (short username oryx)
I20260811 18:50:11.327800 3433 webserver.cc:460] Webserver started at http://127.27.115.252:38383/ using document root <none> and password file <none>
I20260811 18:50:11.328438 3433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.328539 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.328811 3433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:11.331368 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/instance:
uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
format_stamp: "Formatted at 2026-08-11 18:50:11 on dist-test-slave-9jjr"
server_key: "9142347badd0597d1ba7a597f9fb0456"
server_key_iv: "4559796b7fa3b4ba97adaa7382dcdbb4"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.332095 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal/instance:
uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
format_stamp: "Formatted at 2026-08-11 18:50:11 on dist-test-slave-9jjr"
server_key: "9142347badd0597d1ba7a597f9fb0456"
server_key_iv: "4559796b7fa3b4ba97adaa7382dcdbb4"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.336786 3433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:11.339850 3551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.341454 3433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:50:11.341632 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
format_stamp: "Formatted at 2026-08-11 18:50:11 on dist-test-slave-9jjr"
server_key: "9142347badd0597d1ba7a597f9fb0456"
server_key_iv: "4559796b7fa3b4ba97adaa7382dcdbb4"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.341805 3433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.360108 3433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.364331 3433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:11.364529 3433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.375197 3433 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.252:33509
I20260811 18:50:11.375669 3617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.252:33509 every 8 connection(s)
I20260811 18:50:11.376402 3433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
I20260811 18:50:11.379662 3619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:11.384009 3619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:11.385202 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3433
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.252@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
I20260811 18:50:11.400614 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.386960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52358 (local address 127.27.115.254:37959)
0811 18:50:11.387132 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.387138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.387556 (+ 418us) server_negotiation.cc:407] Connection header received
0811 18:50:11.388960 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.388963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.389035 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.389134 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.390054 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.390687 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.391815 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.392059 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.392180 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.395755 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.395760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.395763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.395796 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.397762 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.398507 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.398513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.398518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.398618 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.399147 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.399151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.399153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.399592 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.399696 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.400239 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.400405 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:50:11.401657 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.386782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.27.115.252:33509)
0811 18:50:11.387444 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.387463 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.387504 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:50:11.388674 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.388698 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.388914 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.389124 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.390080 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.391058 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.391959 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.392520 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.392666 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.396006 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.396015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.396035 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.396065 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.398171 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.398821 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.398828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.398830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.398957 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.399375 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.399377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.399378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.399915 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.400138 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.400464 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.400594 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":539,"spinlock_wait_cycles":24448,"thread_start_us":251,"threads_started":1}
I20260811 18:50:11.401773 3622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.386781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52358)
0811 18:50:11.387416 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.387456 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.388706 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.389196 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.389202 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.389592 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.389893 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.389907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.390838 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.390841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.391659 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.391669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.392245 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.392249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.392476 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.393177 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.393202 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.395519 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.397912 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.397921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.397934 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.398396 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.398738 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.398740 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.398742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.398977 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.399718 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.399726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.399922 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.400119 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.400505 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":46,"spinlock_wait_cycles":14208,"thread_start_us":264,"threads_started":1}
I20260811 18:50:11.403045 3619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.252@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
I20260811 18:50:11.412828 3622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.403707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40646)
0811 18:50:11.403915 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.403931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.404003 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.404411 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.404413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.404561 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.404783 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.404790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.405685 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.405688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.406462 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.406473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.407131 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.407135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.407335 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.407803 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.407814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.409236 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.411309 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.411312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.411314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.411600 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.411884 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.411887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.411888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.411967 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.412333 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.412335 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.412427 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.412514 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.412671 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 18:50:11.412873 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.403830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.27.115.253:38273)
0811 18:50:11.404199 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.404203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.404216 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:11.404254 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.404256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.404296 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.404377 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.404897 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.405578 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.406590 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.407024 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.407130 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.409414 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.409417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.409419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.409447 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.411187 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.411713 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.411715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.411717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.411800 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.412050 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.412053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.412054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.412249 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.412327 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.412550 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.412740 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":138,"threads_started":1}
I20260811 18:50:11.413769 3619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.252@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
I20260811 18:50:11.421335 3622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.414360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47496)
0811 18:50:11.414463 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.414476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.414556 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.414732 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.414734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.414875 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.414956 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.414961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.415655 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.415657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.416158 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.416166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.416584 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.416586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.416644 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.417104 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.417115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.418369 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.419858 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.419861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.419862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.420065 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.420397 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.420400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.420401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.420487 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.420913 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.420915 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.421001 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.421084 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.421205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:50:11.421455 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.414421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.27.115.252:33509)
0811 18:50:11.414545 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.414549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.414570 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:50:11.414626 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.414629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.414665 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.414726 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.415051 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.415559 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.416258 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.416484 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.416562 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.418501 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.418504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.418506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.418529 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.419754 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.420156 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.420159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.420160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.420256 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.420612 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.420615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.420616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.420817 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.420909 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.421195 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.421300 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:50:11.425849 3619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Bootstrap starting.
I20260811 18:50:11.428460 3619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:11.429279 3619 log.cc:824] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:11.431676 3619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: No bootstrap required, opened a new log
I20260811 18:50:11.435074 3619 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } }
I20260811 18:50:11.435304 3619 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:11.435339 3619 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec4e8cb5e8ec4610864bfea581b12dce, State: Initialized, Role: FOLLOWER
I20260811 18:50:11.435762 3619 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } }
I20260811 18:50:11.436230 3627 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } } }
I20260811 18:50:11.436384 3627 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:11.437043 3619 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:50:11.442579 3638 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:11.442689 3638 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:11.446698 3433 master_runner.cc:428] Detected that this master ec4e8cb5e8ec4610864bfea581b12dce is joining an existing cluster
I20260811 18:50:11.446775 3433 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.115.252:33509
I20260811 18:50:11.455279 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2845
I20260811 18:50:11.472886 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.27.115.253:38273)
0811 18:50:11.451295 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.451302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.452205 (+ 903us) server_negotiation.cc:407] Connection header received
0811 18:50:11.452417 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.452421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.452488 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.452547 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.453349 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.454224 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.455585 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.455958 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.456036 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.458548 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.458554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.458561 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.458596 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.460727 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.471031 (+ 10304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.471037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.471040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.471169 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.471779 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.471783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.471785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.472042 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.472142 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.472539 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.472707 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:11.473532 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40650)
0811 18:50:11.452113 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.452129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.452289 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.452716 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.452719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.453049 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.453208 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.453217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.454406 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.454411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.455450 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.455462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.456217 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.456220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.456313 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.457773 (+ 1460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.457790 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.458366 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.460863 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.460866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.460868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.470652 (+ 9784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.471567 (+ 915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.471570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.471572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.471666 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.472230 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.472232 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.472339 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.473136 (+ 797us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.473325 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":132,"threads_started":1}
I20260811 18:50:11.475322 3150 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
I20260811 18:50:11.477144 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.27.115.254:37959)
0811 18:50:11.451275 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.451280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.452288 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 18:50:11.452668 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.452671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.452724 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.452781 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.454601 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.455396 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.456752 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.457146 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.457225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.471668 (+ 14443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.471676 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.471682 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.471724 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.474083 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.474849 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.474855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.474860 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.474991 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.475940 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.475947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.475951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.476240 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.476325 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.476755 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.476907 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:50:11.477811 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52368)
0811 18:50:11.452191 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.452204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.452552 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.452938 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.452942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.453483 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.453595 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.453603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.455606 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.455609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.456399 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.456410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.457383 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.457386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.457488 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.470815 (+ 13327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.470840 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.471427 (+ 587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.474404 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.474409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.474414 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.474705 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.475698 (+ 993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.475704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.475709 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.475817 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.476409 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.476414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.476537 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.477340 (+ 803us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.477571 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":96,"threads_started":1}
I20260811 18:50:11.479355 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.27.115.252:33509)
0811 18:50:11.451881 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.451885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.452503 (+ 618us) server_negotiation.cc:407] Connection header received
0811 18:50:11.453151 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.453154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.453194 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.453239 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.456689 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.457481 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.458578 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.458930 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.459003 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.460520 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.460526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.460532 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.460557 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.462087 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.473363 (+ 11276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.473370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.473373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.473560 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.478300 (+ 4740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.478307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.478312 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.478569 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.478667 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.479021 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.479149 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:50:11.479355 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:39833
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:37959
--webserver_port=37889
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:11.479992 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.451764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47504)
0811 18:50:11.452435 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.452447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.453056 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.453747 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.453749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.453932 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.454475 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.454483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.457610 (+ 3127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.457613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.458471 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.458481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.459169 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.459174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.459279 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.459839 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.459882 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.460365 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.462248 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.462252 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.462254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.471444 (+ 9190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.478013 (+ 6569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.478018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.478023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.478176 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.478721 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.478726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.478831 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.479556 (+ 725us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.479731 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":192,"threads_started":1}
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474209, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.254@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
I20260811 18:50:11.491396 3627 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:11.491636 3627 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } }
I20260811 18:50:11.492990 3627 leader_election.cc:288] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2278625c904d828175bedfdac5757b (127.27.115.254:37959), bead63396fa34de09de801712b80edb2 (127.27.115.253:38273)
I20260811 18:50:11.493650 2785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2278625c904d828175bedfdac5757b" is_pre_election: true
I20260811 18:50:11.493651 3483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bead63396fa34de09de801712b80edb2" is_pre_election: true
I20260811 18:50:11.493904 3483 raft_consensus.cc:1799] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Handling vote request from an unknown peer ec4e8cb5e8ec4610864bfea581b12dce
I20260811 18:50:11.494069 3483 raft_consensus.cc:2396] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate ec4e8cb5e8ec4610864bfea581b12dce for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 18:50:11.494657 3553 leader_election.cc:302] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec4e8cb5e8ec4610864bfea581b12dce; no voters: bead63396fa34de09de801712b80edb2, df2278625c904d828175bedfdac5757b
I20260811 18:50:11.495055 3627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:11.496665 3627 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:50:11.497833 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.482633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.27.115.252:33509)
0811 18:50:11.482763 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.482768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.482992 (+ 224us) server_negotiation.cc:407] Connection header received
0811 18:50:11.483107 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.483110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.483171 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.483276 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.483871 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.484660 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.486025 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.486516 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.486639 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.489867 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.489874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.489878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.489917 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.492572 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.495927 (+ 3355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.495935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.495940 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.496074 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.496503 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.496509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.496513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.496933 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.497059 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.497429 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.497599 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:50:11.497835 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.482531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47510)
0811 18:50:11.482805 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.482823 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.482934 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.483323 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.483329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.483576 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.483733 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.483744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.484812 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.484816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.485859 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.485877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.486677 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.486682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.486951 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.487635 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.487656 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.489598 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.492844 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.492852 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.492857 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.495757 (+ 2900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.496230 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.496236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.496241 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.496366 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.497077 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.497080 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.497211 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.497344 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.497648 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 18:50:11.499420 2774 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.115.252:33509: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
member_type: NON_VOTER
last_known_addr {
host: "127.27.115.252"
port: 33509
}
attrs {
promote: true
}
}
dest_uuid: "df2278625c904d828175bedfdac5757b"
cas_config_opid_index: 5
I20260811 18:50:11.500166 2774 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } }
I20260811 18:50:11.501827 3427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.503549 3590 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:50:11.503589 3427 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.115.252:33509
I20260811 18:50:11.503803 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.503904 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:11.504561 3429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:11.505579 3433 master.cc:537] Master@127.27.115.252:33509 shutting down...
W20260811 18:50:11.507644 2760 consensus_peers.cc:487] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b -> Peer ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252:33509): Couldn't send request to peer ec4e8cb5e8ec4610864bfea581b12dce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:50:11.515417 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.502385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.27.115.253:38273)
0811 18:50:11.502515 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.502520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.503162 (+ 642us) server_negotiation.cc:407] Connection header received
0811 18:50:11.503378 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.503381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.503457 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.503538 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:11.504027 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.504873 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.507690 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.509651 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.509899 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.510087 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.515091 (+ 5004us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.515255 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:50:11.516521 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.502059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40662)
0811 18:50:11.503058 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.503094 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.503245 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.503756 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.503758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.503778 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:11.503885 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.503892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.505034 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.505038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.507503 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.507519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.511199 (+ 3680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.511205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.511335 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.511411 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.516241 (+ 4830us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.516376 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917}
I20260811 18:50:11.516553 3433 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:11.517130 3433 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:11.517225 3483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:50:11.517354 3433 tablet_replica.cc:330] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: stopping tablet replica
I20260811 18:50:11.517766 3429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:50:11.519934 3483 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.522274 3650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.522512 3650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:50:11.522532 3522 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df2278625c904d828175bedfdac5757b. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: NON_VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: true } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.522680 3522 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:50:11.528057 3429 raft_consensus.cc:1050] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: attempting to promote NON_VOTER bead63396fa34de09de801712b80edb2 to VOTER
I20260811 18:50:11.528642 3429 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } }
I20260811 18:50:11.530481 3483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 LEARNER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 18:50:11.530522 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.529799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47526)
0811 18:50:11.530329 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.530426 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435}
I20260811 18:50:11.531260 3427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 18:50:11.531630 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.531014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47536)
0811 18:50:11.531320 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.531524 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":114,"threads_started":1}
W20260811 18:50:11.532398 2760 consensus_peers.cc:487] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b -> Peer ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252:33509): Couldn't send request to peer ec4e8cb5e8ec4610864bfea581b12dce. Status: Network error: Client connection negotiation failed: client connection to 127.27.115.252:33509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:50:11.533460 3427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bead63396fa34de09de801712b80edb2 (127.27.115.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.534598 3433 master.cc:552] Master@127.27.115.252:33509 shutdown complete.
I20260811 18:50:11.534644 3433 master_runner.cc:305] Clearing existing system tablet
I20260811 18:50:11.534693 3483 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bead63396fa34de09de801712b80edb2 (127.27.115.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.534909 3433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.534963 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.536382 3522 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.536509 3522 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:11.536731 3433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:11.537731 3653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.537984 3433 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.537974 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.538089 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:11.538108 3433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:11.538164 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
format_stamp: "Formatted at 2026-08-11 18:50:11 on dist-test-slave-9jjr"
server_key: "9142347badd0597d1ba7a597f9fb0456"
server_key_iv: "4559796b7fa3b4ba97adaa7382dcdbb4"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.538345 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.538436 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:11.540496 3433 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:11.544098 3433 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:50:11.544224 3433 log.cc:1196] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 18:50:11.544540 3433 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Deleting consensus metadata
I20260811 18:50:11.544719 3433 master_runner.cc:315] Copying system tablet from 127.27.115.254:37959
I20260811 18:50:11.546063 3433 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: tablet copy: Beginning tablet copy session from remote peer at address 127.27.115.254:37959
I20260811 18:50:11.560590 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.550631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52372)
0811 18:50:11.550990 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.551005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.551135 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.551476 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.551480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.551773 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.551925 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.551933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.553048 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.553052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.554309 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.554325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.554974 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.554978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.555089 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.555762 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.555778 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.556303 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.558710 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.558714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.558716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.559012 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.559352 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.559355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.559357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.559458 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.559936 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.559939 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.560065 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.560186 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.560360 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":93,"threads_started":1}
I20260811 18:50:11.560618 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.550742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.27.115.254:37959)
0811 18:50:11.550887 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.550893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.551096 (+ 203us) server_negotiation.cc:407] Connection header received
0811 18:50:11.551280 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.551283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.551342 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.551431 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.552067 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.552883 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.554476 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.554820 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.554947 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.556567 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.556571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.556574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.556605 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.558571 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.559133 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.559137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.559139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.559244 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.559563 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.559567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.559568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.559826 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.559933 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.560275 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.560414 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:50:11.562249 2796 tablet_copy_service.cc:140] P df2278625c904d828175bedfdac5757b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ec4e8cb5e8ec4610864bfea581b12dce ({username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:52372)
I20260811 18:50:11.562374 2796 tablet_copy_service.cc:161] P df2278625c904d828175bedfdac5757b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ec4e8cb5e8ec4610864bfea581b12dce at {username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:52372: session id = ec4e8cb5e8ec4610864bfea581b12dce-00000000000000000000000000000000
I20260811 18:50:11.563534 2796 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:50:11.564980 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:50:11.567516 3433 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: tablet copy: Starting download of 0 data blocks...
I20260811 18:50:11.567709 3433 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: tablet copy: Starting download of 1 WAL segments...
I20260811 18:50:11.569679 3433 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:50:11.571695 2796 tablet_copy_service.cc:342] P df2278625c904d828175bedfdac5757b: Request end of tablet copy session ec4e8cb5e8ec4610864bfea581b12dce-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:52372
I20260811 18:50:11.571799 2796 tablet_copy_service.cc:433] P df2278625c904d828175bedfdac5757b: ending tablet copy session ec4e8cb5e8ec4610864bfea581b12dce-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ec4e8cb5e8ec4610864bfea581b12dce
I20260811 18:50:11.580377 3433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:11.582778 3663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.583158 3664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.583485 3666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.583896 3433 server_base.cc:1034] running on GCE node
I20260811 18:50:11.584085 3433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:11.584482 3433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:11.585645 3433 hybrid_clock.cc:648] HybridClock initialized: now 1786474211585621 us; error 45 us; skew 500 ppm
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:11.587425 3433 init.cc:377] Logged in from keytab as oryx/127.27.115.252@KRBTEST.COM (short username oryx)
I20260811 18:50:11.587978 3433 webserver.cc:460] Webserver started at http://127.27.115.252:39235/ using document root <none> and password file <none>
I20260811 18:50:11.588192 3433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.588235 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.589708 3433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:11.590893 3678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.591194 3433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:11.591272 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "ec4e8cb5e8ec4610864bfea581b12dce"
format_stamp: "Formatted at 2026-08-11 18:50:11 on dist-test-slave-9jjr"
server_key: "9142347badd0597d1ba7a597f9fb0456"
server_key_iv: "4559796b7fa3b4ba97adaa7382dcdbb4"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.591364 3433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.600826 3433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.604333 3433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.610044 3433 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.252:33509
I20260811 18:50:11.610101 3735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.252:33509 every 8 connection(s)
I20260811 18:50:11.610961 3433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
I20260811 18:50:11.613168 3736 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:11.614356 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Bootstrap starting.
I20260811 18:50:11.630621 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:11.631055 3736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Bootstrap complete.
I20260811 18:50:11.632294 3736 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } }
I20260811 18:50:11.632464 3736 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ec4e8cb5e8ec4610864bfea581b12dce, State: Initialized, Role: LEARNER
I20260811 18:50:11.632577 3736 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } }
I20260811 18:50:11.633004 3741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.633139 3736 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:11.633222 3741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: LEARNER
I20260811 18:50:11.638032 3754 catalog_manager.cc:1259] Loaded cluster ID: 779bc23929264bf68fbbe11102d3abcf
I20260811 18:50:11.638118 3754 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: loading cluster ID for follower catalog manager: success
I20260811 18:50:11.640833 3754 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: acquiring CA information for follower catalog manager: success
I20260811 18:50:11.642825 3754 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:50:11.648756 3648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:11.649097 3648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:11.649189 3648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:11.654217 3648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:11.654358 3648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:11.654459 3648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:11.661139 3648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:39833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=37889
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:11.662691 3648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:11.663987 3648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:11.676156 3761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.676156 3760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.676151 3763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.676751 3648 server_base.cc:1034] running on GCE node
I20260811 18:50:11.677292 3648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:11.678102 3648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:11.679399 3648 hybrid_clock.cc:648] HybridClock initialized: now 1786474211679375 us; error 43 us; skew 500 ppm
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:11.682667 3648 init.cc:377] Logged in from keytab as oryx/127.27.115.193@KRBTEST.COM (short username oryx)
I20260811 18:50:11.683687 3648 webserver.cc:460] Webserver started at http://127.27.115.193:37889/ using document root <none> and password file <none>
I20260811 18:50:11.684302 3648 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.684422 3648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.688640 3648 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:11.691316 3771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.693037 3648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:11.693187 3648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "78d1d62ac15443d1a945d68c8dad4473"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "f6014cda4a0de0048752ee9030804c88"
server_key_iv: "8333b0f0a163b1d9b6a9dc1085b4a292"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.693781 3648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.712553 3648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.715867 3648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:11.716126 3648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.716826 3648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:11.717813 3648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:11.717869 3648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.717913 3648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:11.717969 3648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.733676 3308 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
I20260811 18:50:11.737097 3648 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:39833
I20260811 18:50:11.737246 3898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:39833 every 8 connection(s)
I20260811 18:50:11.738670 3648 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:11.740025 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3648
I20260811 18:50:11.740257 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3001
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
I20260811 18:50:11.770313 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.746001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:57227 (local address 127.27.115.253:38273)
0811 18:50:11.746151 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.746157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.746175 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:50:11.748948 (+ 2773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.748956 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.749069 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.749248 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.752599 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.753895 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.756709 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757279 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.757419 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.762887 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.762893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.762896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.762939 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.765767 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.767691 (+ 1924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.767695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.767697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.767815 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.768547 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.768550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.768553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.768833 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.768930 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.769932 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.770095 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:50:11.770334 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.745781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60867 (local address 127.27.115.254:37959)
0811 18:50:11.745944 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.745949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.749499 (+ 3550us) server_negotiation.cc:407] Connection header received
0811 18:50:11.749766 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.749772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.749848 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.749922 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.752587 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.753468 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.756630 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757009 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.757099 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.764806 (+ 7707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.764822 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.764829 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.764880 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.766849 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.767770 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.767774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.767776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.767908 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.768546 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.768550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.768552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.768836 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.768930 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.769932 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.770095 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:50:11.771147 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:32825
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:37959
--webserver_port=45725
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:11.771632 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.753694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:36351 (local address 127.27.115.252:33509)
0811 18:50:11.755090 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.755095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.755110 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:50:11.755158 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.755161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.755216 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.755309 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:11.756384 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757443 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.758912 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.759381 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.759501 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.763107 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:11.763111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:11.763114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:11.763150 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:11.766670 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.767957 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.767964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.767969 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.768087 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:11.768513 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:11.768519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:11.768524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:11.768839 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:11.768947 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.770969 (+ 2022us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.771138 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":162,"threads_started":1}
I20260811 18:50:11.772217 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.753084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.27.115.193:36351)
0811 18:50:11.754635 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.754655 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.754841 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.755453 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.755456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.755894 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.756175 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.756187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757610 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.757614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.758698 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.758716 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.759645 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.759649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.759849 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.760600 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.760618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.762882 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.767430 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.767436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.767441 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.767739 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.768205 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.768211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.768216 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.768391 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.769013 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.769022 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.769543 (+ 521us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.769831 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.770335 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1355,"mutex_wait_us":130,"spinlock_wait_cycles":896,"thread_start_us":175,"threads_started":1}
I20260811 18:50:11.772487 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.744365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:60867)
0811 18:50:11.749332 (+ 4967us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.749354 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.749615 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.750153 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.750159 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.751043 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.752300 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.752334 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.753857 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.753861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.756466 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.756476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757395 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.757399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.757731 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.758927 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.758942 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.763974 (+ 5032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.766989 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.766992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.766995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.767534 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.768240 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.768243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.768246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.768388 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.769011 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.769019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.769535 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.770580 (+ 1045us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.770841 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4798,"mutex_wait_us":1999,"spinlock_wait_cycles":32128,"thread_start_us":367,"threads_started":1}
I20260811 18:50:11.773463 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.744364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.27.115.193:57227)
0811 18:50:11.745539 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.745644 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.748549 (+ 2905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.749797 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.749817 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.751048 (+ 1231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:11.752302 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.752338 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.754121 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.754168 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.756482 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.756501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.757652 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.757659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.757904 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.758801 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:11.758839 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:11.762455 (+ 3616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:11.766941 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.766952 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.766980 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.767532 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.767952 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:11.767958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:11.767963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:11.768390 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:11.769011 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:11.769019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:11.769535 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.770590 (+ 1055us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.770906 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":391,"spinlock_wait_cycles":116480,"thread_start_us":499,"threads_started":1}
I20260811 18:50:11.774736 3902 heartbeater.cc:347] Connected to a master server at 127.27.115.253:38273
I20260811 18:50:11.774850 3900 heartbeater.cc:347] Connected to a master server at 127.27.115.252:33509
I20260811 18:50:11.775086 3900 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:11.775091 3902 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:11.775195 3899 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:11.775262 3899 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:11.775841 3900 heartbeater.cc:511] Master 127.27.115.252:33509 requested a full tablet report, sending...
I20260811 18:50:11.775839 3902 heartbeater.cc:511] Master 127.27.115.253:38273 requested a full tablet report, sending...
I20260811 18:50:11.775839 3899 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
I20260811 18:50:11.777730 2774 ts_manager.cc:194] Re-registered known tserver with Master: 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833)
I20260811 18:50:11.777941 3479 ts_manager.cc:194] Registered new tserver with Master: 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833)
I20260811 18:50:11.778645 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:60867
I20260811 18:50:11.779037 3688 ts_manager.cc:194] Registered new tserver with Master: 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833)
I20260811 18:50:11.909767 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.905504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47540)
0811 18:50:11.905663 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:11.905681 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:11.905787 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:11.906116 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:11.906118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:11.906178 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:11.906285 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.906291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.907359 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.907361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.908109 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:11.908119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.909126 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.909129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:11.909320 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.909349 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:50:11.909542 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 18:50:11.909642 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 18:50:11.909776 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:11.905614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.27.115.252:33509)
0811 18:50:11.905778 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:11.905786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:11.905806 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:50:11.905871 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:11.905877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:11.905970 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:11.906103 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:11.906421 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.907222 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.908254 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:11.908959 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:11.909098 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:11.909268 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:11.909557 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 18:50:11.909643 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:50:11.909986 3909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:11.910401 3909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:11.910519 3909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:50:11.910882 3699 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 18:50:11.911350 3741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader df2278625c904d828175bedfdac5757b. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:11.911535 3741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: LEARNER
W20260811 18:50:11.915740 3909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:11.915843 3909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:11.915910 3909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:11.922585 3909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:32825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=45725
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:11.923930 3909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:11.925094 3909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:11.936944 3916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.937085 3917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:11.936944 3920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:11.937597 3909 server_base.cc:1034] running on GCE node
I20260811 18:50:11.938104 3909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:11.938983 3909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:11.940189 3909 hybrid_clock.cc:648] HybridClock initialized: now 1786474211940147 us; error 56 us; skew 500 ppm
Aug 11 18:50:11 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:11.943775 3909 init.cc:377] Logged in from keytab as oryx/127.27.115.194@KRBTEST.COM (short username oryx)
I20260811 18:50:11.944909 3909 webserver.cc:460] Webserver started at http://127.27.115.194:45725/ using document root <none> and password file <none>
I20260811 18:50:11.945577 3909 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:11.945719 3909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:11.950680 3909 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:11.953423 3928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.955077 3909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:11.955281 3909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b1bb239449d94043b1a7425c86b760b9"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "0bc9e8b183b99e3913116eaf85e5018c"
server_key_iv: "89f8ce98b9754adbd9cc6d0fa3ad533b"
server_key_version: "encryptionkey@0"
I20260811 18:50:11.955938 3909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:11.983019 3909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:11.987088 3909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:11.987481 3909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:11.988233 3909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:11.989262 3909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:11.989368 3909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:11.989440 3909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:11.989485 3909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:12.003870 3909 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:32825
I20260811 18:50:12.003896 4059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:32825 every 8 connection(s)
I20260811 18:50:12.005071 3909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:12.012394 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 3909
I20260811 18:50:12.012600 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3155
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474211, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
I20260811 18:50:12.026976 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:35015
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:37959
--webserver_port=40179
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:12.027524 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.007288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:33815 (local address 127.27.115.253:38273)
0811 18:50:12.007437 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.007442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.008668 (+ 1226us) server_negotiation.cc:407] Connection header received
0811 18:50:12.010365 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.010368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.010432 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.010492 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.012497 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.013341 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016153 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.016510 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016593 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.021952 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.021957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.021960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.022001 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.024197 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025104 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.025110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.025112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.025235 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025810 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.025816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.025820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.026186 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.026288 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.027159 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.027330 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:12.027628 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.009076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:60793 (local address 127.27.115.252:33509)
0811 18:50:12.009238 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.009244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.009554 (+ 310us) server_negotiation.cc:407] Connection header received
0811 18:50:12.010200 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.010203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.010265 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.010514 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.012505 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.013469 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016279 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.016724 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016828 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.022997 (+ 6169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.023005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.023118 (+ 113us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.023177 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.025142 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025784 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.025788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.025790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.025898 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.026321 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.026328 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.026333 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.026591 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.026673 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.027190 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.027372 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:50:12.028060 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.007361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37211 (local address 127.27.115.254:37959)
0811 18:50:12.007490 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.007495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.008645 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 18:50:12.010385 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.010388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.010443 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.010495 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.012496 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.013344 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016152 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.016510 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016593 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.023408 (+ 6815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.023419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.023423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.023468 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.025774 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.026423 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.026426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.026428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.026529 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.026856 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.026859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.026861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.027140 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.027237 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.027661 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.027831 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:50:12.029769 4066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.007760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:37211)
0811 18:50:12.008433 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.008489 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.009909 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.010816 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.010826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.011628 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.012286 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.012313 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.013653 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.013659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.015982 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.015995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.016836 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.017227 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.018295 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.018320 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.022248 (+ 3928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.025951 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.025956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.026297 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.026642 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.026645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.026647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.026737 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.027343 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.027346 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.027450 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.028218 (+ 768us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.028445 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":572,"spinlock_wait_cycles":8448,"thread_start_us":153,"threads_started":1}
I20260811 18:50:12.029867 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.008948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.27.115.194:60793)
0811 18:50:12.009418 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.009430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.009914 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.010816 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.010824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.011446 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.012287 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.012313 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.013660 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.013666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.016104 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.016128 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.017185 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.017194 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.017407 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.018433 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.018454 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.022752 (+ 4298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.025322 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.025333 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.025644 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.026029 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.026035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.026040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.026198 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.026764 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.026767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.026894 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.027832 (+ 938us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.028361 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":214,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260811 18:50:12.029851 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.007760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.27.115.194:33815)
0811 18:50:12.008548 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.008564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.009909 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.010816 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.010826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.011630 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.012286 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.012313 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.014186 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.014189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.016002 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.016008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.016836 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.016840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.017230 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.018610 (+ 1380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.018629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.021671 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.024376 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.024386 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.024412 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.024950 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.025381 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.025384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.025386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.025655 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.026365 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.026375 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.026817 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.028556 (+ 1739us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.028716 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":472,"spinlock_wait_cycles":59264,"thread_start_us":252,"threads_started":1}
I20260811 18:50:12.031626 4062 heartbeater.cc:347] Connected to a master server at 127.27.115.253:38273
I20260811 18:50:12.031728 4060 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:12.031960 4062 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.032804 4062 heartbeater.cc:511] Master 127.27.115.253:38273 requested a full tablet report, sending...
I20260811 18:50:12.031957 4060 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.031816 4061 heartbeater.cc:347] Connected to a master server at 127.27.115.252:33509
I20260811 18:50:12.033598 4061 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.033710 4060 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
I20260811 18:50:12.033849 4061 heartbeater.cc:511] Master 127.27.115.252:33509 requested a full tablet report, sending...
I20260811 18:50:12.034485 3479 ts_manager.cc:194] Registered new tserver with Master: b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:12.034759 2774 ts_manager.cc:194] Re-registered known tserver with Master: b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:12.034837 3688 ts_manager.cc:194] Registered new tserver with Master: b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:12.035697 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:37211
W20260811 18:50:12.168037 4068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:12.168418 4068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:12.168504 4068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:12.172860 4068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:12.172981 4068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:12.173082 4068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:12.179090 4068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:35015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=40179
--tserver_master_addrs=127.27.115.254:37959,127.27.115.253:38273,127.27.115.252:33509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:12.180430 4068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:12.181594 4068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:12.193979 4082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:12.193928 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:12.194021 4078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:12.194365 4068 server_base.cc:1034] running on GCE node
I20260811 18:50:12.194948 4068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:12.195749 4068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:12.196933 4068 hybrid_clock.cc:648] HybridClock initialized: now 1786474212196896 us; error 52 us; skew 500 ppm
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474212, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:12.201980 4068 init.cc:377] Logged in from keytab as oryx/127.27.115.195@KRBTEST.COM (short username oryx)
I20260811 18:50:12.203383 4068 webserver.cc:460] Webserver started at http://127.27.115.195:40179/ using document root <none> and password file <none>
I20260811 18:50:12.204101 4068 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:12.204237 4068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:12.210521 4068 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:12.213397 4089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:12.215063 4068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:12.215273 4068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "57a994af048149b3b8de443ab4ab7de4"
format_stamp: "Formatted at 2026-08-11 18:50:10 on dist-test-slave-9jjr"
server_key: "56ea4e73eb0b8bb4f980017625171238"
server_key_iv: "11b3bb9894d76c61ba8ff2677726f805"
server_key_version: "encryptionkey@0"
I20260811 18:50:12.215977 4068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:12.256788 4068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:12.261246 4068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:12.261505 4068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:12.262307 4068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:12.263434 4068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:12.263522 4068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:12.263607 4068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:12.263653 4068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:12.275890 4068 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:35015
I20260811 18:50:12.275964 4218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:35015 every 8 connection(s)
I20260811 18:50:12.277391 4068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:12.282256 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 4068
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474212, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.252@KRBTEST.COM
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474212, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
Aug 11 18:50:12 dist-test-slave-9jjr krb5kdc[2724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474212, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.253@KRBTEST.COM
I20260811 18:50:12.303952 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.280284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:34771 (local address 127.27.115.252:33509)
0811 18:50:12.280500 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.280529 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.281915 (+ 1386us) server_negotiation.cc:407] Connection header received
0811 18:50:12.283989 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.283992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.284069 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.284153 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.286651 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.287518 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.290399 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.290796 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.290898 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.297380 (+ 6482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.297385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.297388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.297427 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.299699 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.300569 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.300577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.300582 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.300704 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.301241 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.301248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.301253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.301537 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.301644 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.303472 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.303708 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:50:12.305008 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.283454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:34643 (local address 127.27.115.254:37959)
0811 18:50:12.283729 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.283733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.284289 (+ 556us) server_negotiation.cc:407] Connection header received
0811 18:50:12.284474 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.284477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.284529 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.284581 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.286721 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.287837 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.290357 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.290764 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.290846 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.297319 (+ 6473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.297328 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.297333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.297382 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.300483 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.301075 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.301081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.301083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.301198 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.302152 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.302157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.302159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.302458 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.302563 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.304680 (+ 2117us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.304829 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260811 18:50:12.306056 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.280929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.27.115.195:34771)
0811 18:50:12.281699 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.281771 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.283716 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.284511 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.284523 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.285275 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.286403 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.286433 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.288046 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.288051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.290121 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.290176 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.291094 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.291100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.291490 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.292625 (+ 1135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.292651 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.297105 (+ 4454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.299871 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.299880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.299906 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.300402 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.300837 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.300840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.300843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.301108 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.302562 (+ 1454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.302573 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.303047 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.304280 (+ 1233us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.304764 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":253,"spinlock_wait_cycles":8832,"thread_start_us":275,"threads_started":1}
I20260811 18:50:12.307752 4220 heartbeater.cc:347] Connected to a master server at 127.27.115.252:33509
I20260811 18:50:12.306110 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.282399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:34643)
0811 18:50:12.284203 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.284219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.284365 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.284754 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.284757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.285263 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.286408 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.286432 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.288109 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.288112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.290093 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.290104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.291098 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.291102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.291473 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.292748 (+ 1275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.292770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.297012 (+ 4242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.300653 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.300658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.300661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.300937 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.301327 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.301330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.301332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.301741 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.303890 (+ 2149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.303893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.304011 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.305242 (+ 1231us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.305399 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1734,"mutex_wait_us":203,"spinlock_wait_cycles":5504,"thread_start_us":107,"threads_started":1}
I20260811 18:50:12.308082 4220 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.308837 4220 heartbeater.cc:511] Master 127.27.115.252:33509 requested a full tablet report, sending...
I20260811 18:50:12.309280 4219 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37959
I20260811 18:50:12.309374 4219 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.309585 4219 heartbeater.cc:511] Master 127.27.115.254:37959 requested a full tablet report, sending...
I20260811 18:50:12.310482 3688 ts_manager.cc:194] Registered new tserver with Master: 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:12.310794 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.283586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:54909 (local address 127.27.115.253:38273)
0811 18:50:12.283828 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.283832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.289039 (+ 5207us) server_negotiation.cc:407] Connection header received
0811 18:50:12.289256 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.289259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.289336 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.289393 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.290619 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.291434 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.293105 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.293540 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.293676 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.306310 (+ 12634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.306315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.306318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.306362 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.308295 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.308875 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.308879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.308881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.308988 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.309488 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.309492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.309494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.309747 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.309838 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.310343 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.310572 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"spinlock_wait_cycles":7936}
I20260811 18:50:12.311112 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.282884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.27.115.195:54909)
0811 18:50:12.288892 (+ 6008us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.288914 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.289108 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.289566 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.289569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.290018 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:12.290466 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.290475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.291666 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.291669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.292920 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.292935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.299120 (+ 6185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.299129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.299311 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.301726 (+ 2415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:12.301742 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:12.304408 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:12.308438 (+ 4030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.308443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.308445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.308744 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.309207 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:12.309210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:12.309212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:12.309355 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:12.309940 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:12.309943 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:12.310050 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.310443 (+ 393us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.310617 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5918,"thread_start_us":150,"threads_started":1}
I20260811 18:50:12.311286 2774 ts_manager.cc:194] Re-registered known tserver with Master: 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:12.311790 4223 heartbeater.cc:347] Connected to a master server at 127.27.115.253:38273
I20260811 18:50:12.311887 4223 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:12.312064 4223 heartbeater.cc:511] Master 127.27.115.253:38273 requested a full tablet report, sending...
I20260811 18:50:12.312089 2774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:34643
I20260811 18:50:12.312454 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.287314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.27.115.252:33509)
0811 18:50:12.289690 (+ 2376us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.289695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.294157 (+ 4462us) server_negotiation.cc:407] Connection header received
0811 18:50:12.294302 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.294308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.294378 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.294473 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.295657 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.296489 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.300233 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.300654 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.300784 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.304255 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.304260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.304263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.304296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.308103 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.308811 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.308815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.308817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.308922 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.309555 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.309560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.309563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.309822 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.309971 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.312076 (+ 2105us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.312247 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2282,"thread_start_us":133,"threads_started":1}
I20260811 18:50:12.312858 3479 ts_manager.cc:194] Registered new tserver with Master: 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:12.316205 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.285550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.27.115.253:38273)
0811 18:50:12.288523 (+ 2973us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.288528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.289192 (+ 664us) server_negotiation.cc:407] Connection header received
0811 18:50:12.289248 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.289251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.289336 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.289424 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.295079 (+ 5655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.296145 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.300836 (+ 4691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.301324 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.301452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.310088 (+ 8636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.310092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.310095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.310155 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.314199 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.314757 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.314761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.314763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.314868 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.315213 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.315217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.315219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.315427 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.315595 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.315882 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.316004 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2830,"spinlock_wait_cycles":9856,"thread_start_us":145,"threads_started":1}
I20260811 18:50:12.317855 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.287808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.27.115.254:37959)
0811 18:50:12.288690 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.288695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.290700 (+ 2005us) server_negotiation.cc:407] Connection header received
0811 18:50:12.290761 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.290764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.290824 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.290913 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.292714 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.293590 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.302216 (+ 8626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.302712 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.302862 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.307286 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.307295 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.307299 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.307354 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.315623 (+ 8269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.316194 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.316198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.316199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.316315 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.316799 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.316802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.316804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.317082 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.317254 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.317506 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.317662 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":210,"threads_started":1}
I20260811 18:50:12.321573 2774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:12.325419 2774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:12.329809 3429 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: NON_VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: true } } }
I20260811 18:50:12.330035 3429 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:12.334579 3429 raft_consensus.cc:1050] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: attempting to promote NON_VOTER ec4e8cb5e8ec4610864bfea581b12dce to VOTER
I20260811 18:50:12.335041 3429 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } }
I20260811 18:50:12.336400 3699 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 LEARNER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:50:12.336956 3429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:50:12.337626 3483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Refusing update from remote peer df2278625c904d828175bedfdac5757b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:50:12.338109 3429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:50:12.338882 4247 raft_consensus.cc:2932] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.340394 3699 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.340714 3483 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.341113 4246 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.341255 4246 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:12.341326 4244 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.341423 4244 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:12.341140 4247 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.342198 4247 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:12.342980 4255 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.343169 4255 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:12.343796 3741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2278625c904d828175bedfdac5757b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:12.343940 3741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:12.353564 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.347574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39833 (local address 127.0.0.1:43624)
0811 18:50:12.347742 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.347761 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.347867 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.349499 (+ 1632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.349502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.349526 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:12.349767 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.349776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.351055 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.351061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.351896 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.351908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.353078 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.353085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.353198 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.353224 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.353285 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.353370 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":65}
I20260811 18:50:12.353670 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.347764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.27.115.193:39833)
0811 18:50:12.348165 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.348174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.348215 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:50:12.349074 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.349085 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.349335 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.349713 (+ 378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:12.349877 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.350918 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.352055 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.352916 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.353084 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.353264 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.353341 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.353437 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260811 18:50:12.353611 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.348332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.27.115.195:35015)
0811 18:50:12.348838 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.348877 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.348900 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:50:12.348994 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.349002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.349288 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.349522 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:12.349902 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.351067 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.352166 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.353014 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.353177 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.353321 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.353372 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.353473 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":135,"threads_started":1}
I20260811 18:50:12.353590 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.347809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:35015 (local address 127.0.0.1:59470)
0811 18:50:12.348300 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.348316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.348510 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.349484 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.349487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.349508 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:12.349768 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.349776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.351181 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.351186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.352020 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.352034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.353154 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.353158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.353254 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.353280 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.353341 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.353419 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":51,"thread_start_us":112,"threads_started":1}
I20260811 18:50:12.357254 3823 tablet_service.cc:1467] Processing CreateTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 (DEFAULT_TABLE table=test-table [id=12938ae4c56f4cae97892fbd43c0d681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:12.358078 4257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.347985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:32825 (local address 127.0.0.1:48038)
0811 18:50:12.351067 (+ 3082us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.351084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.352277 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.353615 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.353619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.353639 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:12.353798 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.353808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.355267 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.355271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.356275 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.356285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.357661 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.357664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.357737 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.357756 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.357843 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.357927 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3012,"thread_start_us":128,"threads_started":1}
I20260811 18:50:12.357707 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 (DEFAULT_TABLE table=test-table [id=12938ae4c56f4cae97892fbd43c0d681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:12.358426 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.348474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.27.115.194:32825)
0811 18:50:12.348925 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.348937 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.352188 (+ 3251us) server_negotiation.cc:407] Connection header received
0811 18:50:12.352446 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.352452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.352720 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.352884 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:12.353966 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.355132 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.356692 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.357389 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.357476 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.357575 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.357894 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.357997 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":203,"threads_started":1}
I20260811 18:50:12.358604 3823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:12.358978 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:12.361253 3983 tablet_service.cc:1467] Processing CreateTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 (DEFAULT_TABLE table=test-table [id=12938ae4c56f4cae97892fbd43c0d681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:12.362407 3983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:12.369076 4262 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Bootstrap starting.
I20260811 18:50:12.370195 4263 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Bootstrap starting.
I20260811 18:50:12.372215 4262 tablet_bootstrap.cc:654] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:12.372892 4263 tablet_bootstrap.cc:654] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:12.373373 4264 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Bootstrap starting.
I20260811 18:50:12.374848 4263 log.cc:824] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:12.374919 4262 log.cc:824] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:12.376706 4264 tablet_bootstrap.cc:654] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:12.377528 4262 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: No bootstrap required, opened a new log
I20260811 18:50:12.377528 4263 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: No bootstrap required, opened a new log
I20260811 18:50:12.377767 4263 ts_tablet_manager.cc:1397] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:50:12.377767 4262 ts_tablet_manager.cc:1397] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 18:50:12.378206 4264 log.cc:824] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:12.380686 4264 tablet_bootstrap.cc:492] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: No bootstrap required, opened a new log
I20260811 18:50:12.380909 4264 ts_tablet_manager.cc:1397] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:50:12.383488 4262 raft_consensus.cc:348] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.383913 4262 raft_consensus.cc:374] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:12.383996 4262 raft_consensus.cc:729] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d1d62ac15443d1a945d68c8dad4473, State: Initialized, Role: FOLLOWER
I20260811 18:50:12.384658 4262 consensus_queue.cc:260] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.385138 4263 raft_consensus.cc:348] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.385406 4263 raft_consensus.cc:374] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:12.385458 4263 raft_consensus.cc:729] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a994af048149b3b8de443ab4ab7de4, State: Initialized, Role: FOLLOWER
I20260811 18:50:12.385640 3899 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
I20260811 18:50:12.385900 4262 ts_tablet_manager.cc:1428] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260811 18:50:12.385885 4263 consensus_queue.cc:260] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.387715 4264 raft_consensus.cc:348] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.387988 4264 raft_consensus.cc:374] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:12.388051 4264 raft_consensus.cc:729] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bb239449d94043b1a7425c86b760b9, State: Initialized, Role: FOLLOWER
I20260811 18:50:12.388495 4264 consensus_queue.cc:260] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.388749 4263 ts_tablet_manager.cc:1428] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 18:50:12.389137 4219 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
I20260811 18:50:12.389978 4264 ts_tablet_manager.cc:1428] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 18:50:12.390357 4060 heartbeater.cc:503] Master 127.27.115.254:37959 was elected leader, sending a full tablet report...
W20260811 18:50:12.498118 3903 tablet.cc:2369] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:12.507781 4064 tablet.cc:2369] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:12.529241 4224 tablet.cc:2369] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:12.570053 4270 raft_consensus.cc:482] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:12.570436 4270 raft_consensus.cc:504] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.572220 4270 leader_election.cc:288] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833), b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:12.577792 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.572641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:32825 (local address 127.27.115.195:55189)
0811 18:50:12.572844 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.572861 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.572924 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.573198 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.573200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.573216 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:12.573360 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.573376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.574690 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.574695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.576512 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.576529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.577427 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.577430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.577513 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.577536 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.577603 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.577665 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":308}
I20260811 18:50:12.577877 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.572762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:55189 (local address 127.27.115.194:32825)
0811 18:50:12.572891 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.572896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.572910 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:12.573032 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.573038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.573090 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.573169 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:12.573656 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.574519 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.576700 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.577281 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.577428 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.577501 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.577647 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.577743 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:50:12.578732 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7768d4ba9e5b40cf8d315dd2dcc351a9" candidate_uuid: "57a994af048149b3b8de443ab4ab7de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bb239449d94043b1a7425c86b760b9" is_pre_election: true
I20260811 18:50:12.578853 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.572604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:44017 (local address 127.27.115.193:39833)
0811 18:50:12.572797 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.572802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.572821 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:50:12.572886 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.572890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.572954 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.573052 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:12.573656 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.574529 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.577506 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.578127 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.578302 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.578362 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.578690 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.578755 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:50:12.578897 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.572427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39833 (local address 127.27.115.195:44017)
0811 18:50:12.572625 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:12.572641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:12.572749 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:12.573079 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:12.573081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:12.573097 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:12.573407 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.573417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.574690 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.574695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.576512 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:12.576529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.578337 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.578341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:12.578550 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.578576 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:50:12.578666 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:50:12.578754 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":131}
I20260811 18:50:12.579140 4008 raft_consensus.cc:2454] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57a994af048149b3b8de443ab4ab7de4 in term 0.
I20260811 18:50:12.579733 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7768d4ba9e5b40cf8d315dd2dcc351a9" candidate_uuid: "57a994af048149b3b8de443ab4ab7de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d1d62ac15443d1a945d68c8dad4473" is_pre_election: true
I20260811 18:50:12.579784 4092 leader_election.cc:302] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57a994af048149b3b8de443ab4ab7de4, b1bb239449d94043b1a7425c86b760b9; no voters:
I20260811 18:50:12.580032 3846 raft_consensus.cc:2454] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57a994af048149b3b8de443ab4ab7de4 in term 0.
I20260811 18:50:12.580147 4270 raft_consensus.cc:2781] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:12.580245 4270 raft_consensus.cc:482] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:12.580292 4270 raft_consensus.cc:3037] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:12.581943 4270 raft_consensus.cc:504] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.582401 4270 leader_election.cc:288] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [CANDIDATE]: Term 1 election: Requested vote from peers 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833), b1bb239449d94043b1a7425c86b760b9 (127.27.115.194:32825)
I20260811 18:50:12.582979 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7768d4ba9e5b40cf8d315dd2dcc351a9" candidate_uuid: "57a994af048149b3b8de443ab4ab7de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bb239449d94043b1a7425c86b760b9"
I20260811 18:50:12.583052 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7768d4ba9e5b40cf8d315dd2dcc351a9" candidate_uuid: "57a994af048149b3b8de443ab4ab7de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d1d62ac15443d1a945d68c8dad4473"
I20260811 18:50:12.583190 3846 raft_consensus.cc:3037] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:12.583171 4008 raft_consensus.cc:3037] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:12.585084 4008 raft_consensus.cc:2454] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57a994af048149b3b8de443ab4ab7de4 in term 1.
I20260811 18:50:12.585084 3846 raft_consensus.cc:2454] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57a994af048149b3b8de443ab4ab7de4 in term 1.
I20260811 18:50:12.585562 4092 leader_election.cc:302] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57a994af048149b3b8de443ab4ab7de4, b1bb239449d94043b1a7425c86b760b9; no voters:
I20260811 18:50:12.585809 4270 raft_consensus.cc:2781] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:12.586099 4270 raft_consensus.cc:686] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 1 LEADER]: Becoming Leader. State: Replica: 57a994af048149b3b8de443ab4ab7de4, State: Running, Role: LEADER
I20260811 18:50:12.586668 4270 consensus_queue.cc:237] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } }
I20260811 18:50:12.591487 2773 catalog_manager.cc:5495] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "57a994af048149b3b8de443ab4ab7de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:12.658376 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.652700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.27.115.195:35015)
0811 18:50:12.652868 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.652872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.652968 (+ 96us) server_negotiation.cc:407] Connection header received
0811 18:50:12.653211 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.653216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.653297 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.653410 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:12.654002 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.654688 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.655684 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.655932 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.656099 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.656558 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:12.657337 (+ 779us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:12.657530 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.657708 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.657830 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:50:12.669365 3846 raft_consensus.cc:1261] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Refusing update from remote peer 57a994af048149b3b8de443ab4ab7de4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:12.669366 4008 raft_consensus.cc:1261] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Refusing update from remote peer 57a994af048149b3b8de443ab4ab7de4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:12.670420 4272 consensus_queue.cc:1035] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:12.670653 4270 consensus_queue.cc:1035] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:12.685568 4276 mvcc.cc:205] Tried to move back new op lower bound from 7317398375081476096 to 7317398374759723008. Current Snapshot: MvccSnapshot[applied={T|T < 7317398375081476096}]
I20260811 18:50:12.690443 4278 mvcc.cc:205] Tried to move back new op lower bound from 7317398375081476096 to 7317398374759723008. Current Snapshot: MvccSnapshot[applied={T|T < 7317398375081476096}]
I20260811 18:50:12.695150 2774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:12.695400 2774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:12.703110 2774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df2278625c904d828175bedfdac5757b: Sending DeleteTablet for 3 replicas of tablet 7768d4ba9e5b40cf8d315dd2dcc351a9
I20260811 18:50:12.704413 3983 tablet_service.cc:1514] Processing DeleteTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:12 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:48038
I20260811 18:50:12.704691 3823 tablet_service.cc:1514] Processing DeleteTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:12 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:43624
I20260811 18:50:12.705385 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 7768d4ba9e5b40cf8d315dd2dcc351a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:12 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:59470
I20260811 18:50:12.705479 4288 tablet_replica.cc:330] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: stopping tablet replica
I20260811 18:50:12.706015 4288 raft_consensus.cc:2229] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:12.706622 4288 raft_consensus.cc:2258] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:12.707731 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 2739
I20260811 18:50:12.712430 4288 ts_tablet_manager.cc:1905] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:12.712786 4289 tablet_replica.cc:330] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: stopping tablet replica
I20260811 18:50:12.713369 4289 raft_consensus.cc:2229] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:12.713958 4289 raft_consensus.cc:2258] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:12.716037 4289 ts_tablet_manager.cc:1905] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:12.717049 4288 ts_tablet_manager.cc:1918] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:12.717183 4288 log.cc:1196] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/7768d4ba9e5b40cf8d315dd2dcc351a9
I20260811 18:50:12.717607 4288 ts_tablet_manager.cc:1939] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 78d1d62ac15443d1a945d68c8dad4473: Deleting consensus metadata
I20260811 18:50:12.721429 4289 ts_tablet_manager.cc:1918] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:12.721686 4289 log.cc:1196] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/7768d4ba9e5b40cf8d315dd2dcc351a9
I20260811 18:50:12.722194 4289 ts_tablet_manager.cc:1939] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P 57a994af048149b3b8de443ab4ab7de4: Deleting consensus metadata
I20260811 18:50:12.722476 4287 tablet_replica.cc:330] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: stopping tablet replica
I20260811 18:50:12.723029 4287 raft_consensus.cc:2229] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:12.723579 4287 raft_consensus.cc:2258] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:50:12.724824 4090 connection.cc:535] server connection from 127.0.0.1:59470 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:50:12.725618 3775 connection.cc:535] server connection from 127.0.0.1:43624 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:50:12.727207 4287 ts_tablet_manager.cc:1905] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:12.738960 4287 ts_tablet_manager.cc:1918] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:12.739132 4287 log.cc:1196] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/7768d4ba9e5b40cf8d315dd2dcc351a9
I20260811 18:50:12.739529 4287 ts_tablet_manager.cc:1939] T 7768d4ba9e5b40cf8d315dd2dcc351a9 P b1bb239449d94043b1a7425c86b760b9: Deleting consensus metadata
W20260811 18:50:12.742744 3930 connection.cc:441] server connection from 127.0.0.1:48038 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48038 (request call id 1) could send its response
I20260811 18:50:12.745288 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.730576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47556 (local address 127.27.115.252:33509)
0811 18:50:12.730739 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.730744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.731757 (+ 1013us) server_negotiation.cc:407] Connection header received
0811 18:50:12.732067 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.732070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.732159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.732216 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.735368 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.736396 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.737880 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.738279 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.738370 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.740525 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.740530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.740533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.740566 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.742767 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.743503 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.743507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.743509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.743609 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.743942 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.743946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.743947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.744359 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.744525 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.744960 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.745112 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:50:12.748291 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.734283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.27.115.253:38273)
0811 18:50:12.734432 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.734437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.735017 (+ 580us) server_negotiation.cc:407] Connection header received
0811 18:50:12.735239 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.735243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.735302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.735411 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.736624 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.737465 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.740460 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.740852 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.740940 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.744280 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.744285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.744288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.744325 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.746235 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.746768 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.746772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.746774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.746873 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.747192 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.747196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.747197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.747432 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.747559 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.747972 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.748108 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:12.756217 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.707398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.27.115.253:38273)
0811 18:50:12.707534 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.707539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.722599 (+ 15060us) server_negotiation.cc:407] Connection header received
0811 18:50:12.725467 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.725473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.725554 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.725659 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.727266 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.729932 (+ 2666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.731664 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.732084 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.732188 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.734732 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.734737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.734740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.734779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.736917 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.754689 (+ 17772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.754694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.754696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.754840 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.755175 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.755179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.755180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.755448 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.755586 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.755870 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.756032 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:50:12.766242 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:12.707236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.27.115.252:33509)
0811 18:50:12.707458 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:12.707463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:12.756642 (+ 49179us) server_negotiation.cc:407] Connection header received
0811 18:50:12.756765 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:12.756771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:12.756833 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:12.756931 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:12.757693 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.758326 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.760038 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:12.760316 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:12.760387 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:12.762352 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:12.762356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:12.762360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:12.762400 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:12.764005 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.764564 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.764569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.764571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.764691 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:12.765037 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:12.765040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:12.765042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:12.765285 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:12.765458 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:12.765808 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:50:12.765996 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:50:13.686446 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:13.685835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:55383)
0811 18:50:13.686167 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:13.686283 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
W20260811 18:50:13.686813 3899 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:13.689761 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:13.689068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:57271)
0811 18:50:13.689521 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:13.689625 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":87,"threads_started":1}
W20260811 18:50:13.690152 4060 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:13.691586 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:13.691095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:58077)
0811 18:50:13.691370 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:13.691495 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
W20260811 18:50:13.691932 4219 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.203481 4301 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df2278625c904d828175bedfdac5757b)
I20260811 18:50:14.203662 4301 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } }
I20260811 18:50:14.204264 4302 raft_consensus.cc:482] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader df2278625c904d828175bedfdac5757b)
I20260811 18:50:14.204380 4301 leader_election.cc:288] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df2278625c904d828175bedfdac5757b (127.27.115.254:37959), bead63396fa34de09de801712b80edb2 (127.27.115.253:38273)
I20260811 18:50:14.204785 4302 raft_consensus.cc:504] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } }
I20260811 18:50:14.205328 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.204581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52830)
0811 18:50:14.205078 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.205193 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":84,"threads_started":1}
W20260811 18:50:14.205643 3682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.206219 4302 leader_election.cc:288] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df2278625c904d828175bedfdac5757b (127.27.115.254:37959), ec4e8cb5e8ec4610864bfea581b12dce (127.27.115.252:33509)
I20260811 18:50:14.206789 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.206361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52836)
0811 18:50:14.206603 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.206689 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":61,"threads_started":1}
W20260811 18:50:14.206987 3464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.207700 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.207397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52844)
0811 18:50:14.207558 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.207624 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 18:50:14.207809 3682 leader_election.cc:334] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df2278625c904d828175bedfdac5757b (127.27.115.254:37959): Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.208204 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.207677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52854)
0811 18:50:14.208065 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.208119 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326}
W20260811 18:50:14.208329 3464 leader_election.cc:334] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer df2278625c904d828175bedfdac5757b (127.27.115.254:37959): Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.209539 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.204765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:38273 (local address 127.0.0.1:40692)
0811 18:50:14.205168 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.205185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.205322 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.205718 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.205722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.205745 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.205868 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.205874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.206998 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.207002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.208294 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.208310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.209091 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.209095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.209190 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.209221 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.209309 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.209399 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":109,"threads_started":1}
I20260811 18:50:14.209537 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.204877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.27.115.253:38273)
0811 18:50:14.205231 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.205236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.205256 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:50:14.205514 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.205518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.205573 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.205683 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.206018 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.206843 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.208462 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.208937 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.209085 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.209144 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.209341 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.209416 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":283,"thread_start_us":120,"threads_started":1}
I20260811 18:50:14.210044 3483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bead63396fa34de09de801712b80edb2" is_pre_election: true
I20260811 18:50:14.210287 3483 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec4e8cb5e8ec4610864bfea581b12dce in term 1.
I20260811 18:50:14.210669 3679 leader_election.cc:302] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bead63396fa34de09de801712b80edb2, ec4e8cb5e8ec4610864bfea581b12dce; no voters: df2278625c904d828175bedfdac5757b
I20260811 18:50:14.210909 4301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:50:14.210985 4301 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Starting leader election (detected failure of leader df2278625c904d828175bedfdac5757b)
I20260811 18:50:14.211035 4301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:50:14.211244 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.206702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:33509 (local address 127.0.0.1:47572)
0811 18:50:14.206940 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.206954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.207042 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.207592 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.207598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.207623 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.207718 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.207724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.208823 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.208827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.209781 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.209792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.210648 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.210652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.210741 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.210831 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.210923 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.211023 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
I20260811 18:50:14.211323 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.206836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.27.115.252:33509)
0811 18:50:14.207228 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.207241 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.207258 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:14.207320 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.207323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.207375 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.207473 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.207963 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.208684 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.209912 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.210502 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.210647 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.210703 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.210940 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.211128 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":98,"threads_started":1}
I20260811 18:50:14.212451 4301 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } }
I20260811 18:50:14.212940 4301 leader_election.cc:288] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 election: Requested vote from peers df2278625c904d828175bedfdac5757b (127.27.115.254:37959), bead63396fa34de09de801712b80edb2 (127.27.115.253:38273)
I20260811 18:50:14.213310 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.213053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52862)
0811 18:50:14.213170 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.213241 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:50:14.213465 3483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bead63396fa34de09de801712b80edb2"
I20260811 18:50:14.213631 3483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:50:14.213629 3699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bead63396fa34de09de801712b80edb2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" is_pre_election: true
I20260811 18:50:14.213944 3699 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bead63396fa34de09de801712b80edb2 in current term 2: Already voted for candidate ec4e8cb5e8ec4610864bfea581b12dce in this term.
I20260811 18:50:14.214342 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.214082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52864)
0811 18:50:14.214218 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.214277 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 18:50:14.214476 3682 leader_election.cc:334] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer df2278625c904d828175bedfdac5757b (127.27.115.254:37959): Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
I20260811 18:50:14.214490 3464 leader_election.cc:302] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bead63396fa34de09de801712b80edb2; no voters: df2278625c904d828175bedfdac5757b, ec4e8cb5e8ec4610864bfea581b12dce
I20260811 18:50:14.215041 3483 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec4e8cb5e8ec4610864bfea581b12dce in term 2.
I20260811 18:50:14.215207 4302 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:14.215493 3679 leader_election.cc:302] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bead63396fa34de09de801712b80edb2, ec4e8cb5e8ec4610864bfea581b12dce; no voters: df2278625c904d828175bedfdac5757b
I20260811 18:50:14.215687 4301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:50:14.215952 4301 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [term 2 LEADER]: Becoming Leader. State: Replica: ec4e8cb5e8ec4610864bfea581b12dce, State: Running, Role: LEADER
I20260811 18:50:14.216245 4301 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } }
I20260811 18:50:14.217571 4311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec4e8cb5e8ec4610864bfea581b12dce. Latest consensus state: current_term: 2 leader_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:14.217728 4311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:14.218075 4313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:14.220777 4313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:14.221374 4313 catalog_manager.cc:1259] Loaded cluster ID: 779bc23929264bf68fbbe11102d3abcf
I20260811 18:50:14.221448 4313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:14.222234 4313 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:14.222769 4313 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Loaded TSK: 0
I20260811 18:50:14.223758 4313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:14.286636 3690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:14.291108 3690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:14.295287 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.294964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52880)
0811 18:50:14.295101 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.295181 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:50:14.295738 3483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [term 2 FOLLOWER]: Refusing update from remote peer ec4e8cb5e8ec4610864bfea581b12dce: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 18:50:14.296298 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.295984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.0.0.1:52890)
0811 18:50:14.296132 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.296224 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:50:14.296444 4311 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 18:50:14.296701 3682 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce -> Peer df2278625c904d828175bedfdac5757b (127.27.115.254:37959): Couldn't send request to peer df2278625c904d828175bedfdac5757b. Status: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:50:14.298498 4302 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec4e8cb5e8ec4610864bfea581b12dce. Latest consensus state: current_term: 2 leader_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:14.298696 4302 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:14.300237 4311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:14.300410 4311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:14.300926 4302 sys_catalog.cc:455] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "df2278625c904d828175bedfdac5757b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37959 } } peers { permanent_uuid: "bead63396fa34de09de801712b80edb2" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 38273 } attrs { promote: false } } peers { permanent_uuid: "ec4e8cb5e8ec4610864bfea581b12dce" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 33509 } attrs { promote: false } } }
I20260811 18:50:14.301079 4302 sys_catalog.cc:458] T 00000000000000000000000000000000 P bead63396fa34de09de801712b80edb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:14.314342 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.310401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.27.115.195:35015)
0811 18:50:14.310713 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.310717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.310729 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:50:14.310775 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.310778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.310819 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.310923 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.311176 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.311855 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.313269 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.313706 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.313856 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.313914 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.314085 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.314199 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260811 18:50:14.314353 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.310024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:35015 (local address 127.0.0.1:59480)
0811 18:50:14.310471 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.310484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.310601 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.310931 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.310934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.310951 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.311058 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.311064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.311996 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.311999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.313153 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.313161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.313846 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.313850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.313942 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.313972 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.314067 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.314176 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":38,"thread_start_us":146,"threads_started":1}
I20260811 18:50:14.314683 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.309397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:32825 (local address 127.0.0.1:48040)
0811 18:50:14.309590 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.309610 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.309706 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.310135 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.310138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.310181 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.310329 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.310337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.311634 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.311637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.312881 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.312893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.314269 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.314273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.314370 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.314395 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.314445 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.314521 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":50}
I20260811 18:50:14.315114 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.309715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39833 (local address 127.0.0.1:43628)
0811 18:50:14.310011 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.310027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.310122 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.311032 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.311035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.311055 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.311158 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.311164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.312252 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.312256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.313449 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.313479 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.314674 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.314678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.314782 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.314812 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.314875 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.314974 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":5}
I20260811 18:50:14.315055 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 827baa3640824243b670448f5ed0525f (DEFAULT_TABLE table=test-table [id=0abf0bbed3e045da8551436cf41a65ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:14.315420 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827baa3640824243b670448f5ed0525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:14.315740 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.310046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.27.115.193:39833)
0811 18:50:14.310763 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.310768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.310782 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:14.310838 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.310840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.310888 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.311264 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.311371 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.312117 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.313817 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.314523 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.315382 (+ 859us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.315456 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.315499 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.315586 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":165,"threads_started":1}
I20260811 18:50:14.316107 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.309510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.27.115.194:32825)
0811 18:50:14.309826 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.309830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.309854 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:50:14.309930 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.309934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.309988 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.310463 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.310613 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.311499 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.313036 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.314057 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.315639 (+ 1582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.315692 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.315730 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.315997 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":111,"threads_started":1}
I20260811 18:50:14.316278 3823 tablet_service.cc:1467] Processing CreateTablet for tablet 827baa3640824243b670448f5ed0525f (DEFAULT_TABLE table=test-table [id=0abf0bbed3e045da8551436cf41a65ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:14.316574 3823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827baa3640824243b670448f5ed0525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:14.316694 3983 tablet_service.cc:1467] Processing CreateTablet for tablet 827baa3640824243b670448f5ed0525f (DEFAULT_TABLE table=test-table [id=0abf0bbed3e045da8551436cf41a65ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:14.317011 3983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827baa3640824243b670448f5ed0525f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:14.318393 4325 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Bootstrap starting.
I20260811 18:50:14.319388 4327 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473: Bootstrap starting.
I20260811 18:50:14.319756 4325 tablet_bootstrap.cc:654] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:14.320358 4328 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: Bootstrap starting.
I20260811 18:50:14.320478 4327 tablet_bootstrap.cc:654] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:14.321913 4325 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: No bootstrap required, opened a new log
I20260811 18:50:14.322022 4325 ts_tablet_manager.cc:1397] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:14.322813 4328 tablet_bootstrap.cc:654] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:14.322868 4325 raft_consensus.cc:348] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.323038 4325 raft_consensus.cc:374] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:14.323118 4325 raft_consensus.cc:729] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a994af048149b3b8de443ab4ab7de4, State: Initialized, Role: FOLLOWER
I20260811 18:50:14.323302 4325 consensus_queue.cc:260] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.323448 4327 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473: No bootstrap required, opened a new log
I20260811 18:50:14.323554 4327 ts_tablet_manager.cc:1397] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:14.323721 4325 ts_tablet_manager.cc:1428] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:50:14.324137 4327 raft_consensus.cc:348] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.324275 4327 raft_consensus.cc:374] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:14.324312 4327 raft_consensus.cc:729] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d1d62ac15443d1a945d68c8dad4473, State: Initialized, Role: FOLLOWER
I20260811 18:50:14.324481 4327 consensus_queue.cc:260] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.324908 4327 ts_tablet_manager.cc:1428] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:14.324927 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.324304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:57293)
0811 18:50:14.324711 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.324803 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":112,"threads_started":1}
I20260811 18:50:14.326061 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.325550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:59745)
0811 18:50:14.325880 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.325964 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260811 18:50:14.326746 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.325608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:39383)
0811 18:50:14.326561 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.326625 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":134,"threads_started":1}
I20260811 18:50:14.327075 4220 heartbeater.cc:503] Master 127.27.115.252:33509 was elected leader, sending a full tablet report...
I20260811 18:50:14.329293 4328 tablet_bootstrap.cc:492] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: No bootstrap required, opened a new log
I20260811 18:50:14.329425 4328 ts_tablet_manager.cc:1397] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:50:14.329900 4328 raft_consensus.cc:348] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.330011 4328 raft_consensus.cc:374] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:14.330034 4328 raft_consensus.cc:729] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bb239449d94043b1a7425c86b760b9, State: Initialized, Role: FOLLOWER
I20260811 18:50:14.330232 4328 consensus_queue.cc:260] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.330631 4328 ts_tablet_manager.cc:1428] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:14.331377 3900 heartbeater.cc:503] Master 127.27.115.252:33509 was elected leader, sending a full tablet report...
I20260811 18:50:14.333081 4061 heartbeater.cc:503] Master 127.27.115.252:33509 was elected leader, sending a full tablet report...
I20260811 18:50:14.334448 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.331026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:38013)
0811 18:50:14.333162 (+ 2136us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.334182 (+ 1020us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2036}
I20260811 18:50:14.335012 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.331131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:48137)
0811 18:50:14.334772 (+ 3641us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.334887 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3555,"thread_start_us":145,"threads_started":1}
I20260811 18:50:14.335662 4338 raft_consensus.cc:482] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:14.335793 4338 raft_consensus.cc:504] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.336184 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.335946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:58935)
0811 18:50:14.336055 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.336115 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:50:14.336969 4338 leader_election.cc:288] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833), 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:14.341452 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.337404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:52817 (local address 127.27.115.193:39833)
0811 18:50:14.337563 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.337570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.337591 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:50:14.337650 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.337652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.337696 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.337809 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.338103 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.338729 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.339937 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.340611 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.340771 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.340840 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.341210 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.341299 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:50:14.341452 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.337273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39833 (local address 127.27.115.194:52817)
0811 18:50:14.337382 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.337398 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.337515 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.337821 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.337824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.337845 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.337975 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.337981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.338868 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.338873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.339761 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.339771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.340776 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.340780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.340904 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.340938 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.341213 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.341299 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 18:50:14.342078 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.338063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:54093 (local address 127.27.115.195:35015)
0811 18:50:14.338263 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.338267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.338279 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:50:14.338327 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.338329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.338368 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.338468 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:14.338752 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.339402 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.340814 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.341401 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.341544 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.341617 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.341863 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.341953 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 18:50:14.342156 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827baa3640824243b670448f5ed0525f" candidate_uuid: "b1bb239449d94043b1a7425c86b760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d1d62ac15443d1a945d68c8dad4473" is_pre_election: true
I20260811 18:50:14.342344 3846 raft_consensus.cc:2454] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bb239449d94043b1a7425c86b760b9 in term 0.
I20260811 18:50:14.342523 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.337568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:35015 (local address 127.27.115.194:54093)
0811 18:50:14.337912 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.337928 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.338025 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.338482 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:14.338486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:14.338504 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:14.338631 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.338637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.339523 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.339530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.340676 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:14.340686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.341537 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.341541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:14.341699 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.341724 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:50:14.342286 (+ 562us) client_negotiation.cc:240] Negotiation successful
0811 18:50:14.342372 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":174,"threads_started":1}
I20260811 18:50:14.343016 3930 leader_election.cc:302] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d1d62ac15443d1a945d68c8dad4473, b1bb239449d94043b1a7425c86b760b9; no voters:
I20260811 18:50:14.343122 4165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827baa3640824243b670448f5ed0525f" candidate_uuid: "b1bb239449d94043b1a7425c86b760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a994af048149b3b8de443ab4ab7de4" is_pre_election: true
I20260811 18:50:14.343382 4165 raft_consensus.cc:2454] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bb239449d94043b1a7425c86b760b9 in term 0.
I20260811 18:50:14.343456 4338 raft_consensus.cc:2781] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:14.343550 4338 raft_consensus.cc:482] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:14.343600 4338 raft_consensus.cc:3037] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:14.344830 4338 raft_consensus.cc:504] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.345487 4338 leader_election.cc:288] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [CANDIDATE]: Term 1 election: Requested vote from peers 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833), 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015)
I20260811 18:50:14.346190 3846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827baa3640824243b670448f5ed0525f" candidate_uuid: "b1bb239449d94043b1a7425c86b760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d1d62ac15443d1a945d68c8dad4473"
I20260811 18:50:14.346206 4165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "827baa3640824243b670448f5ed0525f" candidate_uuid: "b1bb239449d94043b1a7425c86b760b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57a994af048149b3b8de443ab4ab7de4"
I20260811 18:50:14.346345 3846 raft_consensus.cc:3037] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:14.346387 4165 raft_consensus.cc:3037] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:14.347470 4165 raft_consensus.cc:2454] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bb239449d94043b1a7425c86b760b9 in term 1.
I20260811 18:50:14.347478 3846 raft_consensus.cc:2454] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bb239449d94043b1a7425c86b760b9 in term 1.
I20260811 18:50:14.347896 3930 leader_election.cc:302] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d1d62ac15443d1a945d68c8dad4473, b1bb239449d94043b1a7425c86b760b9; no voters:
I20260811 18:50:14.348153 4338 raft_consensus.cc:2781] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:14.348440 4338 raft_consensus.cc:686] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 1 LEADER]: Becoming Leader. State: Replica: b1bb239449d94043b1a7425c86b760b9, State: Running, Role: LEADER
I20260811 18:50:14.348773 4338 consensus_queue.cc:237] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } }
I20260811 18:50:14.349300 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.348996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:39249)
0811 18:50:14.349115 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.349187 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:50:14.351948 3690 catalog_manager.cc:5495] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1bb239449d94043b1a7425c86b760b9 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "b1bb239449d94043b1a7425c86b760b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1bb239449d94043b1a7425c86b760b9" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:14.369436 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.365453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.27.115.194:32825)
0811 18:50:14.365585 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:14.365589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:14.365604 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:50:14.365751 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:14.365753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:14.365816 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:14.365920 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:14.366369 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.366883 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.367803 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:14.368026 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:14.368111 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:14.368517 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:14.369031 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:14.369155 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:14.369229 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 18:50:14.369289 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:50:14.373020 3846 raft_consensus.cc:1261] T 827baa3640824243b670448f5ed0525f P 78d1d62ac15443d1a945d68c8dad4473 [term 1 FOLLOWER]: Refusing update from remote peer b1bb239449d94043b1a7425c86b760b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:14.373200 4165 raft_consensus.cc:1261] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Refusing update from remote peer b1bb239449d94043b1a7425c86b760b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:14.373665 4338 consensus_queue.cc:1035] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d1d62ac15443d1a945d68c8dad4473" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:14.373982 4342 consensus_queue.cc:1035] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57a994af048149b3b8de443ab4ab7de4" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:14.377336 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.375602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:59199)
0811 18:50:14.377147 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.377226 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1457}
I20260811 18:50:14.378727 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.378406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:50373)
0811 18:50:14.378534 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.378610 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 18:50:14.380224 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.379873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:39671)
0811 18:50:14.380059 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.380137 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 18:50:14.381903 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.381559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.193:42061)
0811 18:50:14.381767 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.381823 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 18:50:14.383279 4348 mvcc.cc:205] Tried to move back new op lower bound from 7317398382065651712 to 7317398381975715840. Current Snapshot: MvccSnapshot[applied={T|T < 7317398382065651712}]
I20260811 18:50:14.384773 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.384477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.194:40901)
0811 18:50:14.384630 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.384695 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 18:50:14.385653 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.385363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37959 (local address 127.27.115.195:42405)
0811 18:50:14.385511 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.385568 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:37959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:50:14.389072 3690 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:14.389258 3690 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:14.393983 3690 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec4e8cb5e8ec4610864bfea581b12dce: Sending DeleteTablet for 3 replicas of tablet 827baa3640824243b670448f5ed0525f
I20260811 18:50:14.394783 3983 tablet_service.cc:1514] Processing DeleteTablet for tablet 827baa3640824243b670448f5ed0525f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:14 UTC) from {username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:48040
I20260811 18:50:14.395203 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 827baa3640824243b670448f5ed0525f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:14 UTC) from {username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:59480
I20260811 18:50:14.395277 3823 tablet_service.cc:1514] Processing DeleteTablet for tablet 827baa3640824243b670448f5ed0525f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:14 UTC) from {username='oryx', principal='oryx/127.27.115.252@KRBTEST.COM'} at 127.0.0.1:43628
I20260811 18:50:14.395371 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3648
I20260811 18:50:14.410529 4355 tablet_replica.cc:330] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9: stopping tablet replica
I20260811 18:50:14.410795 4355 raft_consensus.cc:2229] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:14.411367 4355 raft_consensus.cc:2258] T 827baa3640824243b670448f5ed0525f P b1bb239449d94043b1a7425c86b760b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:14.412101 4356 tablet_replica.cc:330] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: stopping tablet replica
I20260811 18:50:14.412264 4356 raft_consensus.cc:2229] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:50:14.412300 4303 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:14.411731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39833 (local address 127.0.0.1:43632)
0811 18:50:14.411882 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.411898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:14.412023 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:14.412179 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39833: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 18:50:14.412427 4356 raft_consensus.cc:2258] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:50:14.412482 3680 catalog_manager.cc:4584] TS 78d1d62ac15443d1a945d68c8dad4473 (127.27.115.193:39833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 827baa3640824243b670448f5ed0525f: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39833: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260811 18:50:14.412580 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3909
I20260811 18:50:14.414601 4356 ts_tablet_manager.cc:1905] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:14.416962 4356 ts_tablet_manager.cc:1918] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:14.417030 4356 log.cc:1196] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/827baa3640824243b670448f5ed0525f
I20260811 18:50:14.417299 4356 ts_tablet_manager.cc:1939] T 827baa3640824243b670448f5ed0525f P 57a994af048149b3b8de443ab4ab7de4: Deleting consensus metadata
I20260811 18:50:14.418215 3680 catalog_manager.cc:4841] TS 57a994af048149b3b8de443ab4ab7de4 (127.27.115.195:35015): tablet 827baa3640824243b670448f5ed0525f (table test-table [id=0abf0bbed3e045da8551436cf41a65ca]) successfully deleted
I20260811 18:50:14.424763 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 4068
I20260811 18:50:14.428574 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:14.425098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:32825 (local address 127.0.0.1:48052)
0811 18:50:14.428357 (+ 3259us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:14.428447 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.194:32825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3109}
I20260811 18:50:14.437863 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3314
I20260811 18:50:14.446826 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 3433
2026-08-11T18:50:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6104 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:14 dist-test-slave-9jjr krb5kdc[4360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:14 dist-test-slave-9jjr krb5kdc[4360](info): set up 2 sockets
Aug 11 18:50:14 dist-test-slave-9jjr krb5kdc[4360](info): commencing operation
krb5kdc: starting...
W20260811 18:50:15.594468 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:15 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab.
Entry for principal oryx/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab.
I20260811 18:50:15.737555 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:41493
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab
--principal=oryx/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:15.858870 4374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:15.859251 4374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:15.859340 4374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:15.863708 4374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:15.863832 4374 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:50:15.863867 4374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:15.863914 4374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:15.863955 4374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:15.864012 4374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:15.869853 4374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:41493
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.254.keytab
--principal=oryx/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:15.871179 4374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:15.872386 4374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:15.882021 4381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:15.882350 4374 server_base.cc:1034] running on GCE node
W20260811 18:50:15.882155 4383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:15.882040 4380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:15.883210 4374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:15.884471 4374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:15.885689 4374 hybrid_clock.cc:648] HybridClock initialized: now 1786474215885642 us; error 64 us; skew 500 ppm
Aug 11 18:50:15 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474215, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:15.888639 4374 init.cc:377] Logged in from keytab as oryx/127.27.115.254@KRBTEST.COM (short username oryx)
I20260811 18:50:15.889825 4374 webserver.cc:460] Webserver started at http://127.27.115.254:41521/ using document root <none> and password file <none>
I20260811 18:50:15.890529 4374 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:15.890632 4374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:15.890914 4374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:15.893220 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "cc34075ef0ca494b9acb05371f001134"
format_stamp: "Formatted at 2026-08-11 18:50:15 on dist-test-slave-9jjr"
server_key: "39c9a225491d6e70eabdc02df78deedc"
server_key_iv: "96b1d8f74e66e524b2c6fb68cc1232aa"
server_key_version: "encryptionkey@0"
I20260811 18:50:15.893888 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "cc34075ef0ca494b9acb05371f001134"
format_stamp: "Formatted at 2026-08-11 18:50:15 on dist-test-slave-9jjr"
server_key: "39c9a225491d6e70eabdc02df78deedc"
server_key_iv: "96b1d8f74e66e524b2c6fb68cc1232aa"
server_key_version: "encryptionkey@0"
I20260811 18:50:15.898603 4374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:50:15.901657 4391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:15.903174 4374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:15.903364 4374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "cc34075ef0ca494b9acb05371f001134"
format_stamp: "Formatted at 2026-08-11 18:50:15 on dist-test-slave-9jjr"
server_key: "39c9a225491d6e70eabdc02df78deedc"
server_key_iv: "96b1d8f74e66e524b2c6fb68cc1232aa"
server_key_version: "encryptionkey@0"
I20260811 18:50:15.903508 4374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:15.927409 4374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:15.930887 4374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:15.931136 4374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:15.940168 4374 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:41493
I20260811 18:50:15.940184 4460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:41493 every 8 connection(s)
I20260811 18:50:15.941301 4374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:15.945294 4462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:15.946295 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 4374
I20260811 18:50:15.946476 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:15.946813 28111 external_mini_cluster.cc:1428] Setting key 13e3880f6337445ac097ea07dda7c4f6
I20260811 18:50:15.952736 4462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Bootstrap starting.
I20260811 18:50:15.955667 4462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:15.956468 4462 log.cc:824] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:50:15 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:15.958765 4462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: No bootstrap required, opened a new log
I20260811 18:50:15.963219 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:15.948448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.27.115.254:41493)
0811 18:50:15.948940 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:15.948957 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:15.948996 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:50:15.949892 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:15.949910 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:15.950308 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:15.950606 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:15.951640 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:15.952888 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:15.953835 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:15.954356 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:15.954555 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:15.957273 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:15.957279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:15.957294 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:15.957326 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:15.959765 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:15.960352 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:15.960359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:15.960364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:15.960503 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:15.960822 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:15.960826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:15.960829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:15.961332 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:15.961558 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:15.961895 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 18:50:15.962302 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":157,"threads_started":1}
I20260811 18:50:15.963346 4462 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } }
I20260811 18:50:15.963722 4462 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:15.963832 4462 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc34075ef0ca494b9acb05371f001134, State: Initialized, Role: FOLLOWER
I20260811 18:50:15.964391 4462 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } }
I20260811 18:50:15.964565 4462 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:15.964639 4462 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:15.964771 4462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:15.966341 4462 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } }
I20260811 18:50:15.966779 4462 leader_election.cc:302] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc34075ef0ca494b9acb05371f001134; no voters:
I20260811 18:50:15.967144 4462 leader_election.cc:288] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:15.967301 4467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:15.967600 4467 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [term 1 LEADER]: Becoming Leader. State: Replica: cc34075ef0ca494b9acb05371f001134, State: Running, Role: LEADER
I20260811 18:50:15.968111 4467 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } }
I20260811 18:50:15.968842 4462 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:15.970369 4469 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc34075ef0ca494b9acb05371f001134. Latest consensus state: current_term: 1 leader_uuid: "cc34075ef0ca494b9acb05371f001134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } } }
I20260811 18:50:15.970525 4469 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:15.970844 4468 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc34075ef0ca494b9acb05371f001134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc34075ef0ca494b9acb05371f001134" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 41493 } } }
I20260811 18:50:15.971000 4477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:15.971046 4468 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:15.974109 4477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:15.980372 4477 catalog_manager.cc:1347] Generated new cluster ID: 243d2973a55d4a06ae7350bdd10026e1
I20260811 18:50:15.980480 4477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:15.995625 4477 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:15.997076 4477 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:16.014398 4477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Generated new TSK 0
I20260811 18:50:16.015451 4477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
I20260811 18:50:16.055502 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:41493
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:16.185204 4489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:16.185505 4489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:16.185554 4489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:16.189704 4489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:16.189803 4489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:16.189845 4489 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:50:16.189881 4489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:16.195870 4489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:16.197077 4489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:16.198155 4489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:16.208885 4498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:16.209187 4489 server_base.cc:1034] running on GCE node
W20260811 18:50:16.208890 4496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:16.208921 4495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:16.209844 4489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:16.210728 4489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:16.211937 4489 hybrid_clock.cc:648] HybridClock initialized: now 1786474216211900 us; error 51 us; skew 500 ppm
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:16.215143 4489 init.cc:377] Logged in from keytab as oryx/127.27.115.193@KRBTEST.COM (short username oryx)
I20260811 18:50:16.216300 4489 webserver.cc:460] Webserver started at http://127.27.115.193:46347/ using document root <none> and password file <none>
I20260811 18:50:16.216981 4489 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:16.217170 4489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:16.217445 4489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:16.220024 4489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "36f47d8b920e40918fc10aa454a9312e"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "1a4af0e9c24d0edad51ef04a664cd447"
server_key_iv: "d4fc4c6ec59a39b976bf0442423f7a56"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.220817 4489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "36f47d8b920e40918fc10aa454a9312e"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "1a4af0e9c24d0edad51ef04a664cd447"
server_key_iv: "d4fc4c6ec59a39b976bf0442423f7a56"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.225543 4489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:50:16.228776 4506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.230471 4489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:50:16.230652 4489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "36f47d8b920e40918fc10aa454a9312e"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "1a4af0e9c24d0edad51ef04a664cd447"
server_key_iv: "d4fc4c6ec59a39b976bf0442423f7a56"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.230800 4489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:16.244196 4489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:16.249469 4489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:16.249796 4489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:16.253204 4489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:16.253348 4489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.253446 4489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:16.253499 4489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.268136 4489 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:41109
I20260811 18:50:16.268249 4643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:41109 every 8 connection(s)
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:16.269487 4489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:16.274914 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.254833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60087 (local address 127.27.115.254:41493)
0811 18:50:16.255275 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.255281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.257041 (+ 1760us) server_negotiation.cc:407] Connection header received
0811 18:50:16.259204 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.259208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.259293 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.259407 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.261651 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.262704 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.264955 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.265321 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.265496 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.269302 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.269307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.269309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.269344 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.271649 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.272457 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.272463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.272465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.272703 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.273242 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.273245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.273246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.273543 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.273645 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.274490 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.274689 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260811 18:50:16.275166 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 4489
I20260811 18:50:16.275462 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:16.275820 28111 external_mini_cluster.cc:1428] Setting key 3060dac3e86724f0ff34da604c66fe6d
I20260811 18:50:16.276588 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.255567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.193:60087)
0811 18:50:16.256731 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.256832 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.258849 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.259689 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.259704 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.260408 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.261365 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.261395 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.262929 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.262932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.264727 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.264741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.265580 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.265597 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.265887 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.266858 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.266881 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.268988 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.271810 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.271816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.271829 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.272153 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.272880 (+ 727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.272883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.272884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.273093 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.273693 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.273700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.274034 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.274346 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.275357 (+ 1011us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":213,"threads_started":1}
I20260811 18:50:16.278491 4644 heartbeater.cc:347] Connected to a master server at 127.27.115.254:41493
I20260811 18:50:16.278704 4644 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:16.279421 4644 heartbeater.cc:511] Master 127.27.115.254:41493 requested a full tablet report, sending...
I20260811 18:50:16.281646 4410 ts_manager.cc:194] Registered new tserver with Master: 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109)
I20260811 18:50:16.283526 4410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:60087
WARNING: no policy specified for oryx/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:16.292296 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.281569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.193:58953)
0811 18:50:16.281962 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.281976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.282116 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.282533 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.282535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.282767 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.282907 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.282913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.283860 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.283863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.284587 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.284600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.285248 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.285255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.285410 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.286210 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.286231 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.286891 (+ 660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.290042 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.290046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.290049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.290399 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.290870 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.290874 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.290876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.290990 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.291636 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.291639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.291761 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.291867 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.292030 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":133,"threads_started":1}
I20260811 18:50:16.292348 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.281701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:58953 (local address 127.27.115.254:41493)
0811 18:50:16.281857 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.281862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.282059 (+ 197us) server_negotiation.cc:407] Connection header received
0811 18:50:16.282273 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.282276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.282329 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.282389 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.283051 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.283643 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.284747 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.285076 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.285207 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.287225 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.287231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.287234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.287291 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.289726 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.290565 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.290569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.290570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.290705 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.291186 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.291191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.291193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.291479 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.291615 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.292008 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.292159 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
I20260811 18:50:16.318850 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:41493
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:16.447721 4654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:16.448110 4654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:16.448203 4654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:16.452548 4654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:16.452687 4654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:16.452764 4654 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:50:16.452862 4654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:16.459084 4654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:16.460350 4654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:16.461480 4654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:16.471895 4664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:16.471832 4662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:16.471927 4661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:16.472663 4654 server_base.cc:1034] running on GCE node
I20260811 18:50:16.473075 4654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:16.473747 4654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:16.474951 4654 hybrid_clock.cc:648] HybridClock initialized: now 1786474216474931 us; error 35 us; skew 500 ppm
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:16.478543 4654 init.cc:377] Logged in from keytab as oryx/127.27.115.194@KRBTEST.COM (short username oryx)
I20260811 18:50:16.479756 4654 webserver.cc:460] Webserver started at http://127.27.115.194:36799/ using document root <none> and password file <none>
I20260811 18:50:16.480399 4654 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:16.480500 4654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:16.480751 4654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:16.483042 4654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "cb534b7494b4478e905024d64e934048"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "6459ecf0c8ca1daade9cbe57de3bf77c"
server_key_iv: "629cb5ba35c0843bc009c7cba0efafe5"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.483757 4654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "cb534b7494b4478e905024d64e934048"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "6459ecf0c8ca1daade9cbe57de3bf77c"
server_key_iv: "629cb5ba35c0843bc009c7cba0efafe5"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.487963 4654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:16.490813 4672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.492197 4654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:16.492319 4654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "cb534b7494b4478e905024d64e934048"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "6459ecf0c8ca1daade9cbe57de3bf77c"
server_key_iv: "629cb5ba35c0843bc009c7cba0efafe5"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.492424 4654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:16.525305 4654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:16.528717 4654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:16.528911 4654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:16.530879 4654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:16.530942 4654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.530987 4654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:16.531044 4654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:16.543939 4654 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:42871
I20260811 18:50:16.544095 4808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:42871 every 8 connection(s)
I20260811 18:50:16.545147 4654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:16.547714 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.531810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:40605 (local address 127.27.115.254:41493)
0811 18:50:16.531982 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.531988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.533566 (+ 1578us) server_negotiation.cc:407] Connection header received
0811 18:50:16.534620 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.534625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.534684 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.534743 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.536569 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.537218 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.538740 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.539016 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.539145 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.542764 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.542768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.542771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.542804 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.544757 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.545890 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.545895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.545897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.546004 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.546510 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.546512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.546513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.546736 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.546818 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.547451 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.547574 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:50:16.548632 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 4654
I20260811 18:50:16.548635 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.532725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.194:40605)
0811 18:50:16.533280 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.533339 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.534354 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.535291 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.535303 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.535784 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.536391 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.536428 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.537372 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.537375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.538579 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.538593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.539219 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.539223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.539435 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.540182 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.540200 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.542536 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.544908 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.544913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.544925 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.545738 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.546186 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.546190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.546191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.546383 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.546871 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.546877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.547168 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.547360 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.547655 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260811 18:50:16.548786 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:16.549124 28111 external_mini_cluster.cc:1428] Setting key 4e73c6dae2e03780f4b6947df411dd56
I20260811 18:50:16.550209 4809 heartbeater.cc:347] Connected to a master server at 127.27.115.254:41493
I20260811 18:50:16.550416 4809 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:16.551007 4809 heartbeater.cc:511] Master 127.27.115.254:41493 requested a full tablet report, sending...
I20260811 18:50:16.552245 4410 ts_manager.cc:194] Registered new tserver with Master: cb534b7494b4478e905024d64e934048 (127.27.115.194:42871)
I20260811 18:50:16.553087 4410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:40605
I20260811 18:50:16.561702 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.553319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:36917 (local address 127.27.115.254:41493)
0811 18:50:16.553473 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.553478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.554005 (+ 527us) server_negotiation.cc:407] Connection header received
0811 18:50:16.554093 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.554096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.554195 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.554291 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.554741 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.555464 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.556344 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.556600 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.556690 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.558238 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.558241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.558243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.558265 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.559894 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.560432 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.560434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.560436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.560513 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.560823 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.560825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.560826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.561046 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.561133 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.561470 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.561570 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:50:16.561832 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.553591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.194:36917)
0811 18:50:16.553910 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.553923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.554023 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.554331 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.554333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.554524 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.554627 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.554633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.555626 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.555629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.556215 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.556224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.556745 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.556751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.556881 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.557435 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.557454 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.558014 (+ 560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.560025 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.560031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.560034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.560323 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.560609 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.560614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.560616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.560722 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.561161 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.561167 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.561284 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.561420 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.561594 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for oryx/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
I20260811 18:50:16.586644 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:41493
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:16.710014 4819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:16.710330 4819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:16.710378 4819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:16.714460 4819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:16.714534 4819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:16.714577 4819 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:50:16.714612 4819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:16.720082 4819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.115.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:16.721225 4819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:16.722283 4819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:16.732831 4825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:16.732872 4826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:16.732847 4828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:16.733165 4819 server_base.cc:1034] running on GCE node
I20260811 18:50:16.733741 4819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:16.734655 4819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:16.735826 4819 hybrid_clock.cc:648] HybridClock initialized: now 1786474216735804 us; error 31 us; skew 500 ppm
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:16.738929 4819 init.cc:377] Logged in from keytab as oryx/127.27.115.195@KRBTEST.COM (short username oryx)
I20260811 18:50:16.739903 4819 webserver.cc:460] Webserver started at http://127.27.115.195:42985/ using document root <none> and password file <none>
I20260811 18:50:16.740520 4819 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:16.740584 4819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:16.740818 4819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:16.743222 4819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "deece4d48adf70d3be61bb60d89f760d"
server_key_iv: "452b303dc15c051c224ae8362439747f"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.743934 4819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "deece4d48adf70d3be61bb60d89f760d"
server_key_iv: "452b303dc15c051c224ae8362439747f"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.748734 4819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:16.752158 4835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.754047 4819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:16.754261 4819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
format_stamp: "Formatted at 2026-08-11 18:50:16 on dist-test-slave-9jjr"
server_key: "deece4d48adf70d3be61bb60d89f760d"
server_key_iv: "452b303dc15c051c224ae8362439747f"
server_key_version: "encryptionkey@0"
I20260811 18:50:16.754432 4819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:16.788117 4819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:16.792198 4819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:16.792476 4819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:16.794355 4819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:16.794420 4819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:16.794464 4819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:16.794519 4819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:16.806679 4819 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:39961
I20260811 18:50:16.807060 4971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:39961 every 8 connection(s)
I20260811 18:50:16.807889 4819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:16.810005 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.795580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:54321 (local address 127.27.115.254:41493)
0811 18:50:16.795773 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.795779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.796878 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 18:50:16.798360 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.798364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.798433 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.798539 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.800013 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.800573 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.801946 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.802341 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.802460 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.805515 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.805522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.805527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.805567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.807553 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.808258 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.808262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.808265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.808372 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.808849 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.808855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.808856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.809135 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.809230 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.809748 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.809877 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:50:16.810986 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.796067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.195:54321)
0811 18:50:16.796666 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.796734 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.798016 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.798745 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.798753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.799177 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.799851 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.799867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.800727 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.800730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.801796 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.801809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.802544 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.802548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.802739 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.803513 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.803535 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.805252 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.807715 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.807722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.807743 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.808123 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.808496 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.808499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.808501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.808682 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.809252 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.809257 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.809511 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.809695 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.810006 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":196,"threads_started":1}
I20260811 18:50:16.812320 4972 heartbeater.cc:347] Connected to a master server at 127.27.115.254:41493
I20260811 18:50:16.812510 4972 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:16.813046 4972 heartbeater.cc:511] Master 127.27.115.254:41493 requested a full tablet report, sending...
I20260811 18:50:16.814282 4411 ts_manager.cc:194] Registered new tserver with Master: 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:16.815003 4411 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:54321
I20260811 18:50:16.815606 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 4819
I20260811 18:50:16.815758 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:16.816093 28111 external_mini_cluster.cc:1428] Setting key f4c6cefea0f55af9944b914af2b55c27
I20260811 18:50:16.818295 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:16.824544 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.814883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:56363 (local address 127.27.115.254:41493)
0811 18:50:16.815044 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.815048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.815319 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:50:16.816675 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.816679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.816737 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.816836 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.817445 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.818011 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.819003 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.819392 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.819526 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.820835 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.820842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.820846 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.820886 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.822588 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.823174 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.823177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.823178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.823281 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.823598 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.823601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.823602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.823799 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.823878 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.824298 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.824414 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:50:16.824672 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.814819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.27.115.195:56363)
0811 18:50:16.815212 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.815224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.816502 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.816882 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.816885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.817132 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:16.817257 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.817265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.818177 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.818181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.818870 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.818882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.819523 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.819526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.819603 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.820240 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:16.820253 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:16.820647 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:16.822728 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.822735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.822740 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.823050 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.823378 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:16.823384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:16.823389 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:16.823503 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:16.823911 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:16.823918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:16.824060 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.824191 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.824408 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":178,"threads_started":1}
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:50:16.848846 4465 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:16.841173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.27.115.254:41493)
0811 18:50:16.841324 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.841328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.841341 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:16.841387 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.841389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.841434 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.841506 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.842467 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.843024 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.843935 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.844158 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.844253 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.848664 (+ 4411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34702: BlockingRecv error: recv got EOF from 127.0.0.1:34702 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 18:50:16 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474216, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:16.860838 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.850967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.27.115.254:41493)
0811 18:50:16.851139 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.851143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.851306 (+ 163us) server_negotiation.cc:407] Connection header received
0811 18:50:16.851423 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.851426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.851477 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.851566 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:16.852318 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.852905 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.853821 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.854052 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.854223 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.856970 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:16.856974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:16.856976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:16.857009 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:16.858750 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.859293 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.859297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.859299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.859390 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:16.859719 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:16.859721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:16.859722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:16.859957 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:16.860082 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.860561 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.860701 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:50:16.864336 4411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:16.867794 4411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:16.883836 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.877265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.27.115.194:42871)
0811 18:50:16.878493 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.878505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.878525 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:50:16.878606 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.878612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.878836 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.879489 (+ 653us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:16.879650 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.880439 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.882176 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.883124 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883287 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.883438 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.883617 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.883703 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":102,"threads_started":1}
I20260811 18:50:16.883886 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.877071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42871 (local address 127.0.0.1:34326)
0811 18:50:16.877596 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.877645 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.877806 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.879010 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.879018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.879040 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:16.879304 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.879312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.880555 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.880561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.882021 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.882031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.883280 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.883410 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.883456 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.883579 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.883713 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":38,"thread_start_us":101,"threads_started":1}
I20260811 18:50:16.884382 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.877813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.0.0.1:36684)
0811 18:50:16.878415 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.878430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.878505 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.879304 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.879306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.879324 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:16.879501 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.879508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.882023 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.882026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.882861 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.882870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.883957 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.884094 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.884118 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.884181 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.884246 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260811 18:50:16.884601 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.878038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.27.115.193:41109)
0811 18:50:16.878452 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.878458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.878498 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:50:16.878863 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.878869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.879038 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.879165 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:16.879602 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.880759 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883006 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.883814 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883949 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.884326 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.884377 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.884471 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":111,"threads_started":1}
I20260811 18:50:16.884716 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.877780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.27.115.195:39961)
0811 18:50:16.878202 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:16.878215 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:16.878242 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:50:16.878671 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:16.878676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:16.878894 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:16.879055 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:16.879536 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.880759 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.882320 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.883124 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.883287 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.883438 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:16.884461 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0811 18:50:16.884556 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":112,"threads_started":1}
I20260811 18:50:16.884945 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:16.877252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39961 (local address 127.0.0.1:59198)
0811 18:50:16.878049 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:16.878062 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:16.878223 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:16.879112 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:16.879115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:16.879144 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:16.879411 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.879421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.880896 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.880899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.882209 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:16.882216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:16.884079 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:16.884082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:16.884198 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:16.884222 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:50:16.884307 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:50:16.884384 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":267,"thread_start_us":109,"threads_started":1}
I20260811 18:50:16.887730 4896 tablet_service.cc:1467] Processing CreateTablet for tablet beed2db12c0642358afb087c08cd21f1 (DEFAULT_TABLE table=test-table [id=670d27251bd940b09bdf30dd79e7c71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:16.887732 4567 tablet_service.cc:1467] Processing CreateTablet for tablet beed2db12c0642358afb087c08cd21f1 (DEFAULT_TABLE table=test-table [id=670d27251bd940b09bdf30dd79e7c71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:16.887744 4718 tablet_service.cc:1467] Processing CreateTablet for tablet beed2db12c0642358afb087c08cd21f1 (DEFAULT_TABLE table=test-table [id=670d27251bd940b09bdf30dd79e7c71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:16.888900 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beed2db12c0642358afb087c08cd21f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:16.888911 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beed2db12c0642358afb087c08cd21f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:16.888911 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beed2db12c0642358afb087c08cd21f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:16.896883 5002 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Bootstrap starting.
I20260811 18:50:16.898072 5003 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Bootstrap starting.
I20260811 18:50:16.898535 5004 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Bootstrap starting.
I20260811 18:50:16.900219 5002 tablet_bootstrap.cc:654] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:16.900789 5004 tablet_bootstrap.cc:654] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:16.901129 5002 log.cc:824] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:16.901557 5004 log.cc:824] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:16.903365 5002 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: No bootstrap required, opened a new log
I20260811 18:50:16.903644 5002 ts_tablet_manager.cc:1397] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:50:16.904451 5003 tablet_bootstrap.cc:654] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:16.905507 5003 log.cc:824] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:16.907720 5002 raft_consensus.cc:348] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.908020 5002 raft_consensus.cc:374] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:16.908102 5002 raft_consensus.cc:729] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Initialized, Role: FOLLOWER
I20260811 18:50:16.908365 5004 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: No bootstrap required, opened a new log
I20260811 18:50:16.908672 5002 consensus_queue.cc:260] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.909489 4972 heartbeater.cc:503] Master 127.27.115.254:41493 was elected leader, sending a full tablet report...
I20260811 18:50:16.909924 5002 ts_tablet_manager.cc:1428] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 18:50:16.910043 5003 tablet_bootstrap.cc:492] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: No bootstrap required, opened a new log
I20260811 18:50:16.910359 5004 ts_tablet_manager.cc:1397] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260811 18:50:16.910387 5003 ts_tablet_manager.cc:1397] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 18:50:16.914717 5004 raft_consensus.cc:348] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.914705 5003 raft_consensus.cc:348] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.915067 5003 raft_consensus.cc:374] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:16.915067 5004 raft_consensus.cc:374] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:16.915138 5004 raft_consensus.cc:729] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb534b7494b4478e905024d64e934048, State: Initialized, Role: FOLLOWER
I20260811 18:50:16.915197 5003 raft_consensus.cc:729] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Initialized, Role: FOLLOWER
I20260811 18:50:16.915699 5004 consensus_queue.cc:260] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.915723 5003 consensus_queue.cc:260] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:16.916448 4809 heartbeater.cc:503] Master 127.27.115.254:41493 was elected leader, sending a full tablet report...
I20260811 18:50:16.916448 4644 heartbeater.cc:503] Master 127.27.115.254:41493 was elected leader, sending a full tablet report...
I20260811 18:50:16.917940 5003 ts_tablet_manager.cc:1428] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:50:16.917951 5004 ts_tablet_manager.cc:1428] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260811 18:50:17.022111 4645 tablet.cc:2369] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:17.046561 4810 tablet.cc:2369] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:17.059229 4973 tablet.cc:2369] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:17.142071 5010 raft_consensus.cc:482] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:17.142423 5010 raft_consensus.cc:504] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:17.143934 5010 leader_election.cc:288] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961), 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109)
I20260811 18:50:17.151635 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.144607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:38583 (local address 127.27.115.195:39961)
0811 18:50:17.144869 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.144875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.144990 (+ 115us) server_negotiation.cc:407] Connection header received
0811 18:50:17.145226 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.145230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.145304 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.145418 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:17.145846 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.146845 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.149691 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.150713 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.150897 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.150964 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.151353 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.151447 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 18:50:17.151649 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.145225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:50923 (local address 127.27.115.193:41109)
0811 18:50:17.145420 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.145426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.145525 (+ 99us) server_negotiation.cc:407] Connection header received
0811 18:50:17.145669 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.145672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.145718 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.145816 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:17.146121 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.147002 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.149678 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.150681 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.150897 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.150964 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.151353 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.151447 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:50:17.151762 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.144973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.27.115.194:50923)
0811 18:50:17.145413 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.145431 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.145530 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.145857 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.145860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.145883 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:17.145989 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.145994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.147228 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.147231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.149414 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.149440 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.150899 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.150904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.151146 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.151210 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.151314 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.151487 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":347,"thread_start_us":120,"threads_started":1}
I20260811 18:50:17.151762 5011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.144361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39961 (local address 127.27.115.194:38583)
0811 18:50:17.144791 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.144834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.145043 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.145467 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.145474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.145503 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:17.145707 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.145715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.147193 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.147201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.149414 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.149440 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.150899 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.150904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.151148 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.151208 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.151314 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.151487 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":228,"spinlock_wait_cycles":896,"thread_start_us":118,"threads_started":1}
I20260811 18:50:17.152791 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beed2db12c0642358afb087c08cd21f1" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e" is_pre_election: true
I20260811 18:50:17.152792 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beed2db12c0642358afb087c08cd21f1" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" is_pre_election: true
I20260811 18:50:17.153203 4591 raft_consensus.cc:2454] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:17.153203 4918 raft_consensus.cc:2454] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:17.153900 4678 leader_election.cc:302] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:17.154281 5010 raft_consensus.cc:2781] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:17.154385 5010 raft_consensus.cc:482] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:17.154420 5010 raft_consensus.cc:3037] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.155644 5010 raft_consensus.cc:504] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:17.156113 5010 leader_election.cc:288] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Requested vote from peers 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961), 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109)
I20260811 18:50:17.157058 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beed2db12c0642358afb087c08cd21f1" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
I20260811 18:50:17.157058 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beed2db12c0642358afb087c08cd21f1" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e"
I20260811 18:50:17.157243 4591 raft_consensus.cc:3037] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.157244 4918 raft_consensus.cc:3037] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.158923 4591 raft_consensus.cc:2454] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:17.159324 4918 raft_consensus.cc:2454] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:17.159545 4678 leader_election.cc:302] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f47d8b920e40918fc10aa454a9312e, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:17.159816 5010 raft_consensus.cc:2781] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:17.160043 5010 raft_consensus.cc:686] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Becoming Leader. State: Replica: cb534b7494b4478e905024d64e934048, State: Running, Role: LEADER
I20260811 18:50:17.160425 5010 consensus_queue.cc:237] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } }
I20260811 18:50:17.164255 4411 catalog_manager.cc:5495] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb534b7494b4478e905024d64e934048 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "cb534b7494b4478e905024d64e934048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:17.188388 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.184405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.27.115.194:42871)
0811 18:50:17.184548 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.184551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.184565 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:17.184738 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.184741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.184818 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.184949 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:17.185158 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.185677 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.186696 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.186933 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.187024 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.187331 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.187819 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.187912 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.188093 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.188194 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:50:17.195549 4591 raft_consensus.cc:1261] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.195549 4918 raft_consensus.cc:1261] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.196484 5013 consensus_queue.cc:1035] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.196707 5010 consensus_queue.cc:1035] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.205641 5016 mvcc.cc:205] Tried to move back new op lower bound from 7317398393624358912 to 7317398393492078592. Current Snapshot: MvccSnapshot[applied={T|T < 7317398393624358912}]
I20260811 18:50:17.217909 4411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:17.218170 4411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:17.221395 4411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Sending DeleteTablet for 3 replicas of tablet beed2db12c0642358afb087c08cd21f1
I20260811 18:50:17.222496 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet beed2db12c0642358afb087c08cd21f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:17 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34326
I20260811 18:50:17.222769 4896 tablet_service.cc:1514] Processing DeleteTablet for tablet beed2db12c0642358afb087c08cd21f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:17 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:59198
I20260811 18:50:17.223353 5027 tablet_replica.cc:330] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: stopping tablet replica
I20260811 18:50:17.223388 4567 tablet_service.cc:1514] Processing DeleteTablet for tablet beed2db12c0642358afb087c08cd21f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:17 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36684
I20260811 18:50:17.223296 4411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:50:17.223783 5027 raft_consensus.cc:2229] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:17.223944 5028 tablet_replica.cc:330] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: stopping tablet replica
W20260811 18:50:17.224221 4411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:17.224282 5028 raft_consensus.cc:2229] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:17.224308 5027 raft_consensus.cc:2258] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:17.224803 5029 tablet_replica.cc:330] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: stopping tablet replica
I20260811 18:50:17.224838 5028 raft_consensus.cc:2258] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:17.225301 5029 raft_consensus.cc:2229] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:17.225633 5029 raft_consensus.cc:2258] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:17.227396 5029 ts_tablet_manager.cc:1905] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:17.227612 5028 ts_tablet_manager.cc:1905] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:17.230875 5028 ts_tablet_manager.cc:1918] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:17.230870 5029 ts_tablet_manager.cc:1918] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:17.231016 5028 log.cc:1196] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/beed2db12c0642358afb087c08cd21f1
I20260811 18:50:17.231017 5029 log.cc:1196] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/beed2db12c0642358afb087c08cd21f1
I20260811 18:50:17.231380 5028 ts_tablet_manager.cc:1939] T beed2db12c0642358afb087c08cd21f1 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting consensus metadata
I20260811 18:50:17.231401 5029 ts_tablet_manager.cc:1939] T beed2db12c0642358afb087c08cd21f1 P 36f47d8b920e40918fc10aa454a9312e: Deleting consensus metadata
I20260811 18:50:17.231921 4567 tablet_service.cc:1467] Processing CreateTablet for tablet a279a563577241ccbf32758f79b56da7 (DEFAULT_TABLE table=test-table [id=6f15bdc1af464656a296032bde337c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:17.232223 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a279a563577241ccbf32758f79b56da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.233305 4896 tablet_service.cc:1467] Processing CreateTablet for tablet a279a563577241ccbf32758f79b56da7 (DEFAULT_TABLE table=test-table [id=6f15bdc1af464656a296032bde337c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:17.233507 5027 ts_tablet_manager.cc:1905] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:17.233659 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a279a563577241ccbf32758f79b56da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.235160 4395 catalog_manager.cc:4841] TS 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961): tablet beed2db12c0642358afb087c08cd21f1 (table test-table [id=670d27251bd940b09bdf30dd79e7c71b]) successfully deleted
I20260811 18:50:17.235560 5003 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Bootstrap starting.
I20260811 18:50:17.236521 5002 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Bootstrap starting.
I20260811 18:50:17.236836 5003 tablet_bootstrap.cc:654] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.237494 4718 tablet_service.cc:1467] Processing CreateTablet for tablet a279a563577241ccbf32758f79b56da7 (DEFAULT_TABLE table=test-table [id=6f15bdc1af464656a296032bde337c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:17.237632 4395 catalog_manager.cc:4841] TS 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109): tablet beed2db12c0642358afb087c08cd21f1 (table test-table [id=670d27251bd940b09bdf30dd79e7c71b]) successfully deleted
I20260811 18:50:17.237713 5002 tablet_bootstrap.cc:654] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.237850 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a279a563577241ccbf32758f79b56da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.239248 5003 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: No bootstrap required, opened a new log
I20260811 18:50:17.239353 5003 ts_tablet_manager.cc:1397] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:17.239622 5027 ts_tablet_manager.cc:1918] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:17.239742 5027 log.cc:1196] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/beed2db12c0642358afb087c08cd21f1
I20260811 18:50:17.240113 5027 ts_tablet_manager.cc:1939] T beed2db12c0642358afb087c08cd21f1 P cb534b7494b4478e905024d64e934048: Deleting consensus metadata
I20260811 18:50:17.240278 5003 raft_consensus.cc:348] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.240420 5003 raft_consensus.cc:374] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.240460 5003 raft_consensus.cc:729] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.240621 5003 consensus_queue.cc:260] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.240933 5003 ts_tablet_manager.cc:1428] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:17.241310 5004 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Bootstrap starting.
I20260811 18:50:17.241597 4393 catalog_manager.cc:4841] TS cb534b7494b4478e905024d64e934048 (127.27.115.194:42871): tablet beed2db12c0642358afb087c08cd21f1 (table test-table [id=670d27251bd940b09bdf30dd79e7c71b]) successfully deleted
I20260811 18:50:17.242501 5004 tablet_bootstrap.cc:654] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.242887 5002 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: No bootstrap required, opened a new log
I20260811 18:50:17.243000 5002 ts_tablet_manager.cc:1397] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260811 18:50:17.243556 5002 raft_consensus.cc:348] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.243691 5002 raft_consensus.cc:374] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.243724 5002 raft_consensus.cc:729] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.243868 5002 consensus_queue.cc:260] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.244165 5002 ts_tablet_manager.cc:1428] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:17.245765 5004 tablet_bootstrap.cc:492] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: No bootstrap required, opened a new log
I20260811 18:50:17.245872 5004 ts_tablet_manager.cc:1397] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:50:17.246502 5004 raft_consensus.cc:348] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.246655 5004 raft_consensus.cc:374] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.246723 5004 raft_consensus.cc:729] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb534b7494b4478e905024d64e934048, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.246873 5004 consensus_queue.cc:260] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.247615 5004 ts_tablet_manager.cc:1428] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:17.629940 5010 raft_consensus.cc:482] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:17.630151 5010 raft_consensus.cc:504] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.630621 5010 leader_election.cc:288] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:17.631256 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a279a563577241ccbf32758f79b56da7" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e" is_pre_election: true
I20260811 18:50:17.631265 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a279a563577241ccbf32758f79b56da7" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" is_pre_election: true
I20260811 18:50:17.631414 4591 raft_consensus.cc:2454] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:17.631474 4918 raft_consensus.cc:2454] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:17.631836 4678 leader_election.cc:302] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f47d8b920e40918fc10aa454a9312e, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:17.632071 5010 raft_consensus.cc:2781] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:17.632139 5010 raft_consensus.cc:482] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:17.632180 5010 raft_consensus.cc:3037] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.633239 5010 raft_consensus.cc:504] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.633644 5010 leader_election.cc:288] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Requested vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:17.634195 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a279a563577241ccbf32758f79b56da7" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
I20260811 18:50:17.634208 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a279a563577241ccbf32758f79b56da7" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e"
I20260811 18:50:17.634351 4918 raft_consensus.cc:3037] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.634379 4591 raft_consensus.cc:3037] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.635457 4918 raft_consensus.cc:2454] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:17.635557 4591 raft_consensus.cc:2454] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:17.635830 4678 leader_election.cc:302] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:17.636044 5010 raft_consensus.cc:2781] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:17.636138 5010 raft_consensus.cc:686] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Becoming Leader. State: Replica: cb534b7494b4478e905024d64e934048, State: Running, Role: LEADER
I20260811 18:50:17.636332 5010 consensus_queue.cc:237] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.638337 4411 catalog_manager.cc:5495] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb534b7494b4478e905024d64e934048 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "cb534b7494b4478e905024d64e934048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:50:17 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474215, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.254@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:17.743155 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.732446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:41493 (local address 127.0.0.1:34726)
0811 18:50:17.732890 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.732936 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.733091 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.733444 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.733447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.733910 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:17.734026 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.734034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.735070 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.735073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.735793 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.735808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.736677 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.736680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.736752 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.737401 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:17.737415 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:17.739238 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:17.741450 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:17.741467 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:17.741478 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:17.741754 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:17.742042 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:17.742045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:17.742047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:17.742184 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:17.742621 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:17.742627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:17.742750 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.742852 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.742998 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":159,"threads_started":1}
I20260811 18:50:17.743220 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.732470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.27.115.254:41493)
0811 18:50:17.732979 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.732990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.733011 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:50:17.733250 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.733252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.733304 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.733396 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:17.734250 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.734922 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.736066 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.736473 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.736610 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.739445 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:17.739449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:17.739451 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:17.739478 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:17.741278 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:17.741853 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:17.741855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:17.741857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:17.741943 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:17.742303 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:17.742306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:17.742307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:17.742514 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:17.742596 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.742913 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.743077 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":421,"thread_start_us":159,"threads_started":1}
I20260811 18:50:17.747794 4411 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34726:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:50:17.748721 4411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:17.754783 4567 tablet_service.cc:1467] Processing CreateTablet for tablet 42acc93859c04ef4bcea588cfd68c821 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4cef7c072924458bad5f724de5bac48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:50:17.754913 4896 tablet_service.cc:1467] Processing CreateTablet for tablet 42acc93859c04ef4bcea588cfd68c821 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4cef7c072924458bad5f724de5bac48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:50:17.755056 4718 tablet_service.cc:1467] Processing CreateTablet for tablet 42acc93859c04ef4bcea588cfd68c821 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4cef7c072924458bad5f724de5bac48]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:50:17.755267 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42acc93859c04ef4bcea588cfd68c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.755273 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42acc93859c04ef4bcea588cfd68c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.755409 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42acc93859c04ef4bcea588cfd68c821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:17.759195 5047 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: Bootstrap starting.
I20260811 18:50:17.759189 5044 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: Bootstrap starting.
I20260811 18:50:17.759626 5046 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: Bootstrap starting.
I20260811 18:50:17.760210 5044 tablet_bootstrap.cc:654] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.760318 5047 tablet_bootstrap.cc:654] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.760653 5046 tablet_bootstrap.cc:654] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:17.761606 5047 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: No bootstrap required, opened a new log
I20260811 18:50:17.761716 5047 ts_tablet_manager.cc:1397] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:17.761951 5046 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: No bootstrap required, opened a new log
I20260811 18:50:17.762065 5046 ts_tablet_manager.cc:1397] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:17.762372 5047 raft_consensus.cc:348] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.762537 5047 raft_consensus.cc:374] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.762600 5047 raft_consensus.cc:729] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb534b7494b4478e905024d64e934048, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.762567 5046 raft_consensus.cc:348] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.762701 5046 raft_consensus.cc:374] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.762749 5046 raft_consensus.cc:729] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.762755 5047 consensus_queue.cc:260] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.762840 5044 tablet_bootstrap.cc:492] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: No bootstrap required, opened a new log
I20260811 18:50:17.762928 5044 ts_tablet_manager.cc:1397] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:17.762883 5046 consensus_queue.cc:260] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.763104 5047 ts_tablet_manager.cc:1428] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:17.763031 5010 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.763186 5010 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.763242 5046 ts_tablet_manager.cc:1428] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:17.763870 5044 raft_consensus.cc:348] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.763875 5052 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.764029 5052 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.764097 5044 raft_consensus.cc:374] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:17.764199 5044 raft_consensus.cc:729] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Initialized, Role: FOLLOWER
I20260811 18:50:17.764451 5044 consensus_queue.cc:260] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.764887 5044 ts_tablet_manager.cc:1428] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:17.764883 5053 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.765070 5053 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.876077 5053 raft_consensus.cc:482] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:17.876281 5053 raft_consensus.cc:504] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.877534 5053 leader_election.cc:288] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb534b7494b4478e905024d64e934048 (127.27.115.194:42871), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:17.883383 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.877929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39961 (local address 127.27.115.193:35983)
0811 18:50:17.878375 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.878389 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.878536 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.878855 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.878858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.878888 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:17.879101 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.879106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.880246 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.880250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.882081 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.882101 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.882924 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.882927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.883019 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.883045 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.883124 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.883195 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":203,"spinlock_wait_cycles":35968,"thread_start_us":161,"threads_started":1}
I20260811 18:50:17.883427 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.878018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:35983 (local address 127.27.115.195:39961)
0811 18:50:17.878333 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.878337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.878558 (+ 221us) server_negotiation.cc:407] Connection header received
0811 18:50:17.878647 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.878650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.878706 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.878817 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:17.879267 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.880094 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.882297 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.882782 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.882911 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.882960 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.883174 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.883261 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":116,"threads_started":1}
I20260811 18:50:17.884311 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42acc93859c04ef4bcea588cfd68c821" candidate_uuid: "36f47d8b920e40918fc10aa454a9312e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" is_pre_election: true
I20260811 18:50:17.884541 4918 raft_consensus.cc:2454] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36f47d8b920e40918fc10aa454a9312e in term 0.
I20260811 18:50:17.884621 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.877831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42871 (local address 127.27.115.193:34845)
0811 18:50:17.878353 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.878370 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.878536 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.878855 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.878859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.878889 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:17.879097 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.879105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.880246 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.880250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.882081 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.882101 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.884014 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.884018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.884166 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.884194 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.884287 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.884391 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":285,"spinlock_wait_cycles":22912,"thread_start_us":123,"threads_started":1}
I20260811 18:50:17.884629 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.878015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:34845 (local address 127.27.115.194:42871)
0811 18:50:17.878295 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.878300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.878558 (+ 258us) server_negotiation.cc:407] Connection header received
0811 18:50:17.878647 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.878650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.878717 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.878817 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:17.879247 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.880086 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.883038 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.883746 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.883902 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.883982 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.884330 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.884421 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
I20260811 18:50:17.885105 4512 leader_election.cc:302] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, 36f47d8b920e40918fc10aa454a9312e; no voters:
I20260811 18:50:17.885497 5053 raft_consensus.cc:2781] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:17.885583 5053 raft_consensus.cc:482] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:17.885639 5053 raft_consensus.cc:3037] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.885812 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42acc93859c04ef4bcea588cfd68c821" candidate_uuid: "36f47d8b920e40918fc10aa454a9312e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb534b7494b4478e905024d64e934048" is_pre_election: true
I20260811 18:50:17.886113 4755 raft_consensus.cc:2454] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36f47d8b920e40918fc10aa454a9312e in term 0.
I20260811 18:50:17.886969 5053 raft_consensus.cc:504] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.887419 5053 leader_election.cc:288] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [CANDIDATE]: Term 1 election: Requested vote from peers cb534b7494b4478e905024d64e934048 (127.27.115.194:42871), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:17.887817 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42acc93859c04ef4bcea588cfd68c821" candidate_uuid: "36f47d8b920e40918fc10aa454a9312e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb534b7494b4478e905024d64e934048"
I20260811 18:50:17.887850 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42acc93859c04ef4bcea588cfd68c821" candidate_uuid: "36f47d8b920e40918fc10aa454a9312e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
I20260811 18:50:17.887976 4755 raft_consensus.cc:3037] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.887998 4918 raft_consensus.cc:3037] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:17.889310 4755 raft_consensus.cc:2454] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36f47d8b920e40918fc10aa454a9312e in term 1.
I20260811 18:50:17.889310 4918 raft_consensus.cc:2454] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36f47d8b920e40918fc10aa454a9312e in term 1.
I20260811 18:50:17.889712 4512 leader_election.cc:302] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, 36f47d8b920e40918fc10aa454a9312e; no voters:
I20260811 18:50:17.889956 5053 raft_consensus.cc:2781] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:17.890295 5053 raft_consensus.cc:686] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [term 1 LEADER]: Becoming Leader. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Running, Role: LEADER
I20260811 18:50:17.890588 5053 consensus_queue.cc:237] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:17.891988 5058 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: TxnStatusTablet state changed. Reason: New leader 36f47d8b920e40918fc10aa454a9312e. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.892117 5058 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:50:17.892441 5060 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:50:17.892613 5060 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:50:17.893435 4410 catalog_manager.cc:5495] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e reported cstate change: term changed from 0 to 1, leader changed from <none> to 36f47d8b920e40918fc10aa454a9312e (127.27.115.193). New cstate: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:17.926455 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.921925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.27.115.193:41109)
0811 18:50:17.922350 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.922357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.922370 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:17.922452 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.922454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.922510 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.922656 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:17.922956 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.923662 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.924601 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.924865 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.924991 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.925390 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.925904 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.926044 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.926207 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.926275 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":225,"threads_started":1}
I20260811 18:50:17.926438 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.921781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.0.0.1:36692)
0811 18:50:17.921980 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.921996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.922107 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.922672 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.922676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.922684 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:50:17.922814 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.922821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.923815 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.923819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.924455 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.924465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.925006 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.925011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.925128 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.925186 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.926019 (+ 833us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.926042 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.926170 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.926285 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 18:50:17.930305 4918 raft_consensus.cc:1261] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Refusing update from remote peer 36f47d8b920e40918fc10aa454a9312e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.930500 4755 raft_consensus.cc:1261] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Refusing update from remote peer 36f47d8b920e40918fc10aa454a9312e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.930956 5053 consensus_queue.cc:1035] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.931134 5058 consensus_queue.cc:1035] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.933146 5010 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: TxnStatusTablet state changed. Reason: New leader 36f47d8b920e40918fc10aa454a9312e. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.933274 5010 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.933548 5052 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: TxnStatusTablet state changed. Reason: New leader 36f47d8b920e40918fc10aa454a9312e. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.933676 5052 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.936085 5053 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.936252 5053 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:50:17.936367 5053 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.936431 5053 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 36f47d8b920e40918fc10aa454a9312e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:50:17.937619 5010 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.937785 5010 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P cb534b7494b4478e905024d64e934048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.938398 5064 mvcc.cc:205] Tried to move back new op lower bound from 7317398396636475392 to 7317398396483006464. Current Snapshot: MvccSnapshot[applied={T|T < 7317398396636475392 or (T in {7317398396636475392})}]
I20260811 18:50:17.939350 5052 tablet_replica.cc:439] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36f47d8b920e40918fc10aa454a9312e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } } }
I20260811 18:50:17.939482 5052 tablet_replica.cc:442] T 42acc93859c04ef4bcea588cfd68c821 P 1e2f848977314ed5984a6c1aaf4bcae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:50:17.942227 5065 mvcc.cc:205] Tried to move back new op lower bound from 7317398396636475392 to 7317398396483006464. Current Snapshot: MvccSnapshot[applied={T|T < 7317398396636475392}]
I20260811 18:50:17.957527 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.951625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:52525 (local address 127.27.115.193:41109)
0811 18:50:17.951764 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.951769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.954190 (+ 2421us) server_negotiation.cc:407] Connection header received
0811 18:50:17.954425 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.954429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.954494 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.954587 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:17.954865 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.955415 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.956344 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.956591 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.956715 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.957037 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.957127 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.957209 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.957308 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.957366 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:50:17.957530 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.951517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.27.115.194:52525)
0811 18:50:17.954035 (+ 2518us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.954051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.954283 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.954600 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.954603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.954612 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:50:17.954738 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.954751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.955545 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.955548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.956202 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.956222 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.956724 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.956728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.956817 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.956884 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.957224 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.957233 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.957297 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.957392 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2382,"thread_start_us":145,"threads_started":1}
I20260811 18:50:17.965963 4591 raft_consensus.cc:1261] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.965970 4918 raft_consensus.cc:1261] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:17.966456 5013 consensus_queue.cc:1035] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.966820 5013 consensus_queue.cc:1035] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:17.991264 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.987292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42871 (local address 127.27.115.193:35341)
0811 18:50:17.987771 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:17.987787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:17.987899 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:17.988132 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:17.988135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:17.988143 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:50:17.988259 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.988264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.989219 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.989224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.989942 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:17.989952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.990489 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.990493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:17.990582 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.990641 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.990964 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.990974 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:50:17.991039 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:50:17.991125 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":116,"threads_started":1}
I20260811 18:50:17.991253 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:17.987428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:35341 (local address 127.27.115.194:42871)
0811 18:50:17.987566 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:17.987571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:17.987870 (+ 299us) server_negotiation.cc:407] Connection header received
0811 18:50:17.987980 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:17.987982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:17.988030 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:17.988093 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:17.988411 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.989117 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.990071 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:17.990371 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:17.990487 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:17.990784 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:17.990869 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:17.990956 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:17.991049 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 18:50:17.991105 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:50:18.014071 4410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.014289 4410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.016557 4410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Sending DeleteTablet for 3 replicas of tablet a279a563577241ccbf32758f79b56da7
I20260811 18:50:18.017303 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet a279a563577241ccbf32758f79b56da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34326
I20260811 18:50:18.017729 5082 tablet_replica.cc:330] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: stopping tablet replica
I20260811 18:50:18.017894 5082 raft_consensus.cc:2229] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:18.017904 4896 tablet_service.cc:1514] Processing DeleteTablet for tablet a279a563577241ccbf32758f79b56da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:59198
I20260811 18:50:18.018165 5082 raft_consensus.cc:2258] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.018934 5082 ts_tablet_manager.cc:1905] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.022056 4567 tablet_service.cc:1514] Processing DeleteTablet for tablet a279a563577241ccbf32758f79b56da7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36684
I20260811 18:50:18.022173 5082 ts_tablet_manager.cc:1918] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:50:18.022261 5082 log.cc:1196] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/a279a563577241ccbf32758f79b56da7
I20260811 18:50:18.022511 5085 tablet_replica.cc:330] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: stopping tablet replica
I20260811 18:50:18.022547 5082 ts_tablet_manager.cc:1939] T a279a563577241ccbf32758f79b56da7 P cb534b7494b4478e905024d64e934048: Deleting consensus metadata
I20260811 18:50:18.022650 5085 raft_consensus.cc:2229] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:18.022791 5085 raft_consensus.cc:2258] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.023610 5085 ts_tablet_manager.cc:1905] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.024063 4393 catalog_manager.cc:4841] TS cb534b7494b4478e905024d64e934048 (127.27.115.194:42871): tablet a279a563577241ccbf32758f79b56da7 (table test-table [id=6f15bdc1af464656a296032bde337c18]) successfully deleted
I20260811 18:50:18.025943 5083 tablet_replica.cc:330] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: stopping tablet replica
I20260811 18:50:18.026160 5083 raft_consensus.cc:2229] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:18.026307 5083 raft_consensus.cc:2258] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.027200 5083 ts_tablet_manager.cc:1905] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.027737 5085 ts_tablet_manager.cc:1918] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:50:18.027833 5085 log.cc:1196] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/a279a563577241ccbf32758f79b56da7
I20260811 18:50:18.028131 5085 ts_tablet_manager.cc:1939] T a279a563577241ccbf32758f79b56da7 P 36f47d8b920e40918fc10aa454a9312e: Deleting consensus metadata
I20260811 18:50:18.029160 4395 catalog_manager.cc:4841] TS 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109): tablet a279a563577241ccbf32758f79b56da7 (table test-table [id=6f15bdc1af464656a296032bde337c18]) successfully deleted
I20260811 18:50:18.029831 5083 ts_tablet_manager.cc:1918] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:50:18.029937 5083 log.cc:1196] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/a279a563577241ccbf32758f79b56da7
I20260811 18:50:18.030277 5083 ts_tablet_manager.cc:1939] T a279a563577241ccbf32758f79b56da7 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting consensus metadata
I20260811 18:50:18.031343 4395 catalog_manager.cc:4841] TS 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961): tablet a279a563577241ccbf32758f79b56da7 (table test-table [id=6f15bdc1af464656a296032bde337c18]) successfully deleted
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[4360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:50:18.051946 5042 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:18.043910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.27.115.254:41493)
0811 18:50:18.044079 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.044085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.044292 (+ 207us) server_negotiation.cc:407] Connection header received
0811 18:50:18.044467 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.044471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.044527 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.044613 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:18.045707 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.046406 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.047672 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.047918 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.048005 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.051810 (+ 3805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34734: BlockingRecv error: recv got EOF from 127.0.0.1:34734 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[4360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM
I20260811 18:50:18.063334 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.053796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.27.115.254:41493)
0811 18:50:18.053940 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.053945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.054047 (+ 102us) server_negotiation.cc:407] Connection header received
0811 18:50:18.054237 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.054240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.054298 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.054388 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:18.055026 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.055563 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.056421 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.056662 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.056800 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.059758 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:18.059763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:18.059766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:18.059809 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:18.061415 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:18.061898 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:18.061902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:18.061904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:18.061995 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:18.062327 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:18.062331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:18.062333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:18.062556 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:18.062707 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.062987 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.063158 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:50:18.066411 4410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:18.066919 4410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:18.072650 4896 tablet_service.cc:1467] Processing CreateTablet for tablet caca32c80c864feba94dacfd283c3a55 (DEFAULT_TABLE table=test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.072779 4567 tablet_service.cc:1467] Processing CreateTablet for tablet caca32c80c864feba94dacfd283c3a55 (DEFAULT_TABLE table=test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.073006 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caca32c80c864feba94dacfd283c3a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.073081 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caca32c80c864feba94dacfd283c3a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.075210 5046 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Bootstrap starting.
I20260811 18:50:18.075510 5044 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Bootstrap starting.
I20260811 18:50:18.075802 4718 tablet_service.cc:1467] Processing CreateTablet for tablet caca32c80c864feba94dacfd283c3a55 (DEFAULT_TABLE table=test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.076098 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caca32c80c864feba94dacfd283c3a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.076319 5046 tablet_bootstrap.cc:654] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.076426 5044 tablet_bootstrap.cc:654] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.077792 5044 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: No bootstrap required, opened a new log
I20260811 18:50:18.077900 5044 ts_tablet_manager.cc:1397] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:18.078418 5044 raft_consensus.cc:348] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.078559 5044 raft_consensus.cc:374] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.078593 5044 raft_consensus.cc:729] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.078748 5044 consensus_queue.cc:260] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.078886 5047 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Bootstrap starting.
I20260811 18:50:18.079051 5044 ts_tablet_manager.cc:1428] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:18.079783 5047 tablet_bootstrap.cc:654] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.080593 5046 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: No bootstrap required, opened a new log
I20260811 18:50:18.080690 5046 ts_tablet_manager.cc:1397] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:50:18.081284 5046 raft_consensus.cc:348] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.081430 5046 raft_consensus.cc:374] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.081465 5046 raft_consensus.cc:729] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.081656 5046 consensus_queue.cc:260] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.082614 5046 ts_tablet_manager.cc:1428] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:18.083055 5047 tablet_bootstrap.cc:492] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: No bootstrap required, opened a new log
I20260811 18:50:18.083177 5047 ts_tablet_manager.cc:1397] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:18.083637 5047 raft_consensus.cc:348] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.083739 5047 raft_consensus.cc:374] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.083761 5047 raft_consensus.cc:729] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb534b7494b4478e905024d64e934048, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.083899 5047 consensus_queue.cc:260] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.084221 5047 ts_tablet_manager.cc:1428] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:18.354238 5080 raft_consensus.cc:482] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:18.354442 5080 raft_consensus.cc:504] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.354923 5080 leader_election.cc:288] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:18.355947 4917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caca32c80c864feba94dacfd283c3a55" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" is_pre_election: true
I20260811 18:50:18.355948 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caca32c80c864feba94dacfd283c3a55" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e" is_pre_election: true
I20260811 18:50:18.356161 4917 raft_consensus.cc:2454] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:18.356178 4591 raft_consensus.cc:2454] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 0.
I20260811 18:50:18.356670 4678 leader_election.cc:302] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f47d8b920e40918fc10aa454a9312e, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:18.356962 5080 raft_consensus.cc:2781] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:18.357029 5080 raft_consensus.cc:482] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:18.357060 5080 raft_consensus.cc:3037] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.358462 5080 raft_consensus.cc:504] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.358999 5080 leader_election.cc:288] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Requested vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961)
I20260811 18:50:18.359670 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caca32c80c864feba94dacfd283c3a55" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e"
I20260811 18:50:18.359863 4591 raft_consensus.cc:3037] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.359875 4917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caca32c80c864feba94dacfd283c3a55" candidate_uuid: "cb534b7494b4478e905024d64e934048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2f848977314ed5984a6c1aaf4bcae9"
I20260811 18:50:18.360069 4917 raft_consensus.cc:3037] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.361074 4591 raft_consensus.cc:2454] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:18.361301 4917 raft_consensus.cc:2454] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb534b7494b4478e905024d64e934048 in term 1.
I20260811 18:50:18.361616 4678 leader_election.cc:302] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f47d8b920e40918fc10aa454a9312e, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:18.361887 5080 raft_consensus.cc:2781] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:18.362020 5080 raft_consensus.cc:686] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Becoming Leader. State: Replica: cb534b7494b4478e905024d64e934048, State: Running, Role: LEADER
I20260811 18:50:18.362236 5080 consensus_queue.cc:237] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.364513 4410 catalog_manager.cc:5495] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb534b7494b4478e905024d64e934048 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "cb534b7494b4478e905024d64e934048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:18.384868 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.381425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.27.115.194:42871)
0811 18:50:18.381588 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.381592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.381611 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:50:18.381663 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.381665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.381713 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.381802 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:18.382187 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.382738 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.383665 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.383892 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.383979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.384334 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.384427 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.384513 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.384617 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.384671 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:50:18.387341 4591 raft_consensus.cc:1261] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:18.387341 4917 raft_consensus.cc:1261] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Refusing update from remote peer cb534b7494b4478e905024d64e934048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:18.387928 5080 consensus_queue.cc:1035] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:18.388152 5013 consensus_queue.cc:1035] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:18.396432 4411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.396610 4411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.398702 4411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Sending DeleteTablet for 3 replicas of tablet caca32c80c864feba94dacfd283c3a55
I20260811 18:50:18.399356 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet caca32c80c864feba94dacfd283c3a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34326
I20260811 18:50:18.399449 4896 tablet_service.cc:1514] Processing DeleteTablet for tablet caca32c80c864feba94dacfd283c3a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:59198
I20260811 18:50:18.399559 5082 tablet_replica.cc:330] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: stopping tablet replica
I20260811 18:50:18.399680 4567 tablet_service.cc:1514] Processing DeleteTablet for tablet caca32c80c864feba94dacfd283c3a55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36684
I20260811 18:50:18.399745 5082 raft_consensus.cc:2229] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:18.399919 5085 tablet_replica.cc:330] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: stopping tablet replica
I20260811 18:50:18.399919 5083 tablet_replica.cc:330] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: stopping tablet replica
I20260811 18:50:18.399988 5082 raft_consensus.cc:2258] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.400048 5085 raft_consensus.cc:2229] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:18.400063 5083 raft_consensus.cc:2229] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:18.400200 5085 raft_consensus.cc:2258] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.400224 5083 raft_consensus.cc:2258] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.400447 4411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:50:18.400789 5082 ts_tablet_manager.cc:1905] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.400892 5085 ts_tablet_manager.cc:1905] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.401016 5083 ts_tablet_manager.cc:1905] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:50:18.401234 4411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:18.403462 5082 ts_tablet_manager.cc:1918] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:18.403462 5085 ts_tablet_manager.cc:1918] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:18.403551 5085 log.cc:1196] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/caca32c80c864feba94dacfd283c3a55
I20260811 18:50:18.403551 5082 log.cc:1196] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/caca32c80c864feba94dacfd283c3a55
I20260811 18:50:18.403767 5083 ts_tablet_manager.cc:1918] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:18.403834 5085 ts_tablet_manager.cc:1939] T caca32c80c864feba94dacfd283c3a55 P 36f47d8b920e40918fc10aa454a9312e: Deleting consensus metadata
I20260811 18:50:18.403859 5083 log.cc:1196] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/caca32c80c864feba94dacfd283c3a55
I20260811 18:50:18.403870 5082 ts_tablet_manager.cc:1939] T caca32c80c864feba94dacfd283c3a55 P cb534b7494b4478e905024d64e934048: Deleting consensus metadata
I20260811 18:50:18.404145 5083 ts_tablet_manager.cc:1939] T caca32c80c864feba94dacfd283c3a55 P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting consensus metadata
I20260811 18:50:18.404744 4393 catalog_manager.cc:4841] TS cb534b7494b4478e905024d64e934048 (127.27.115.194:42871): tablet caca32c80c864feba94dacfd283c3a55 (table test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]) successfully deleted
I20260811 18:50:18.404757 4395 catalog_manager.cc:4841] TS 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109): tablet caca32c80c864feba94dacfd283c3a55 (table test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]) successfully deleted
I20260811 18:50:18.405028 4395 catalog_manager.cc:4841] TS 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961): tablet caca32c80c864feba94dacfd283c3a55 (table test-table [id=4a1caf9bea614c58b3271b6d174b7d5f]) successfully deleted
I20260811 18:50:18.408030 4567 tablet_service.cc:1467] Processing CreateTablet for tablet fa2eec9b23ea4afab9f6c356da54fecd (DEFAULT_TABLE table=test-table [id=c5fe8cc00ada46a29a10bdaeabffe4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.408058 4718 tablet_service.cc:1467] Processing CreateTablet for tablet fa2eec9b23ea4afab9f6c356da54fecd (DEFAULT_TABLE table=test-table [id=c5fe8cc00ada46a29a10bdaeabffe4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.408150 4896 tablet_service.cc:1467] Processing CreateTablet for tablet fa2eec9b23ea4afab9f6c356da54fecd (DEFAULT_TABLE table=test-table [id=c5fe8cc00ada46a29a10bdaeabffe4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:18.408360 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2eec9b23ea4afab9f6c356da54fecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.408357 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2eec9b23ea4afab9f6c356da54fecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.408406 4896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2eec9b23ea4afab9f6c356da54fecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:18.410383 5047 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Bootstrap starting.
I20260811 18:50:18.410810 5044 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e: Bootstrap starting.
I20260811 18:50:18.411291 5047 tablet_bootstrap.cc:654] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.411656 5044 tablet_bootstrap.cc:654] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.412292 5046 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Bootstrap starting.
I20260811 18:50:18.412590 5047 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: No bootstrap required, opened a new log
I20260811 18:50:18.412693 5047 ts_tablet_manager.cc:1397] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:18.412927 5044 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e: No bootstrap required, opened a new log
I20260811 18:50:18.413017 5044 ts_tablet_manager.cc:1397] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:18.413110 5047 raft_consensus.cc:348] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.413203 5047 raft_consensus.cc:374] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.413223 5047 raft_consensus.cc:729] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb534b7494b4478e905024d64e934048, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.413375 5047 consensus_queue.cc:260] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.413426 5046 tablet_bootstrap.cc:654] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:18.413472 5044 raft_consensus.cc:348] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.413594 5044 raft_consensus.cc:374] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.413631 5044 raft_consensus.cc:729] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f47d8b920e40918fc10aa454a9312e, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.413669 5047 ts_tablet_manager.cc:1428] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:18.413789 5044 consensus_queue.cc:260] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.414088 5044 ts_tablet_manager.cc:1428] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:18.414933 5046 tablet_bootstrap.cc:492] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: No bootstrap required, opened a new log
I20260811 18:50:18.415028 5046 ts_tablet_manager.cc:1397] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:18.415527 5046 raft_consensus.cc:348] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.415666 5046 raft_consensus.cc:374] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:18.415723 5046 raft_consensus.cc:729] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Initialized, Role: FOLLOWER
I20260811 18:50:18.415895 5046 consensus_queue.cc:260] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.416627 5046 ts_tablet_manager.cc:1428] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:18.478696 5052 raft_consensus.cc:482] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:18.478915 5052 raft_consensus.cc:504] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.480094 5052 leader_election.cc:288] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), cb534b7494b4478e905024d64e934048 (127.27.115.194:42871)
I20260811 18:50:18.484586 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.480562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:34559 (local address 127.27.115.194:42871)
0811 18:50:18.480738 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.480743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.480983 (+ 240us) server_negotiation.cc:407] Connection header received
0811 18:50:18.481186 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.481189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.481233 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.481326 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:18.481660 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.482264 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.483497 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.483938 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.484081 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.484143 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.484349 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.484430 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:18.484608 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.480447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42871 (local address 127.27.115.195:34559)
0811 18:50:18.480868 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:18.480883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:18.481063 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:18.481344 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:18.481347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:18.481366 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:18.481539 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.481545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.482391 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.482396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.483342 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.483354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.484082 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.484085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.484196 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.484220 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:50:18.484332 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 18:50:18.484422 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"spinlock_wait_cycles":30464,"thread_start_us":114,"threads_started":1}
I20260811 18:50:18.484632 5111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.480447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.27.115.195:42507)
0811 18:50:18.480874 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:18.480889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:18.481062 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:18.481360 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:18.481363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:18.481382 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:18.481546 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.481552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.482567 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.482570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.483383 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.483393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.484109 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.484113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.484243 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.484271 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:50:18.484360 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:50:18.484441 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":82,"spinlock_wait_cycles":8832,"thread_start_us":85,"threads_started":1}
I20260811 18:50:18.484618 5113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.480671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:42507 (local address 127.27.115.193:41109)
0811 18:50:18.480906 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.480912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.480993 (+ 81us) server_negotiation.cc:407] Connection header received
0811 18:50:18.481179 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.481182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.481250 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.481349 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:18.481666 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.482415 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.483522 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.483969 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.484109 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.484174 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.484395 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.484482 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260811 18:50:18.485222 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2eec9b23ea4afab9f6c356da54fecd" candidate_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb534b7494b4478e905024d64e934048" is_pre_election: true
I20260811 18:50:18.485430 4755 raft_consensus.cc:2454] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e2f848977314ed5984a6c1aaf4bcae9 in term 0.
I20260811 18:50:18.485507 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2eec9b23ea4afab9f6c356da54fecd" candidate_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e" is_pre_election: true
I20260811 18:50:18.485702 4591 raft_consensus.cc:2454] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e2f848977314ed5984a6c1aaf4bcae9 in term 0.
I20260811 18:50:18.485949 4837 leader_election.cc:302] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, cb534b7494b4478e905024d64e934048; no voters:
I20260811 18:50:18.486351 5052 raft_consensus.cc:2781] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:18.486423 5052 raft_consensus.cc:482] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:18.486462 5052 raft_consensus.cc:3037] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.487608 5052 raft_consensus.cc:504] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.488019 5052 leader_election.cc:288] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [CANDIDATE]: Term 1 election: Requested vote from peers 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109), cb534b7494b4478e905024d64e934048 (127.27.115.194:42871)
I20260811 18:50:18.488487 4591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2eec9b23ea4afab9f6c356da54fecd" candidate_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f47d8b920e40918fc10aa454a9312e"
I20260811 18:50:18.488492 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2eec9b23ea4afab9f6c356da54fecd" candidate_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb534b7494b4478e905024d64e934048"
I20260811 18:50:18.488648 4591 raft_consensus.cc:3037] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.488662 4755 raft_consensus.cc:3037] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:18.489674 4591 raft_consensus.cc:2454] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e2f848977314ed5984a6c1aaf4bcae9 in term 1.
I20260811 18:50:18.489676 4755 raft_consensus.cc:2454] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e2f848977314ed5984a6c1aaf4bcae9 in term 1.
I20260811 18:50:18.490077 4839 leader_election.cc:302] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e2f848977314ed5984a6c1aaf4bcae9, 36f47d8b920e40918fc10aa454a9312e; no voters:
I20260811 18:50:18.490343 5052 raft_consensus.cc:2781] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:18.490607 5052 raft_consensus.cc:686] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 LEADER]: Becoming Leader. State: Replica: 1e2f848977314ed5984a6c1aaf4bcae9, State: Running, Role: LEADER
I20260811 18:50:18.490929 5052 consensus_queue.cc:237] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } }
I20260811 18:50:18.493737 4411 catalog_manager.cc:5495] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e2f848977314ed5984a6c1aaf4bcae9" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39961 } health_report { overall_health: HEALTHY } } }
I20260811 18:50:18.528903 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.524886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.27.115.195:39961)
0811 18:50:18.525189 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.525197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.525210 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:18.525291 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.525293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.525352 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.525478 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:18.525785 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.526480 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.527295 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.527536 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.527634 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.527948 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.528433 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.528525 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.528686 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.528742 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":143,"threads_started":1}
I20260811 18:50:18.541901 5113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.537917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:60281 (local address 127.27.115.193:41109)
0811 18:50:18.538087 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.538091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.538289 (+ 198us) server_negotiation.cc:407] Connection header received
0811 18:50:18.538396 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.538399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.538447 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.538520 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:18.538734 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.539252 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.540566 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.540792 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.540892 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.541301 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.541392 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.541499 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.541695 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.541773 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:50:18.541905 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.537814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.27.115.195:60281)
0811 18:50:18.538199 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:18.538213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:18.538318 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:18.538545 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:18.538547 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:18.538553 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:50:18.538663 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.538667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.539375 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.539378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.540419 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.540432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.540939 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.540946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.541035 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.541120 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.541550 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.541558 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:50:18.541634 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:50:18.541739 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":124,"threads_started":1}
I20260811 18:50:18.551126 4591 raft_consensus.cc:1261] T fa2eec9b23ea4afab9f6c356da54fecd P 36f47d8b920e40918fc10aa454a9312e [term 1 FOLLOWER]: Refusing update from remote peer 1e2f848977314ed5984a6c1aaf4bcae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:18.551492 4755 raft_consensus.cc:1261] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Refusing update from remote peer 1e2f848977314ed5984a6c1aaf4bcae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:18.551785 5052 consensus_queue.cc:1035] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f47d8b920e40918fc10aa454a9312e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:18.552162 5114 consensus_queue.cc:1035] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb534b7494b4478e905024d64e934048" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:18.573618 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.570002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:45825 (local address 127.27.115.195:39961)
0811 18:50:18.570174 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:18.570179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:18.570194 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:50:18.570423 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:18.570426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:18.570488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:18.570574 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:18.570850 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.571431 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.572434 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.572753 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.572879 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.573146 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.573254 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.573337 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:18.573450 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 18:50:18.573503 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:50:18.573632 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.569651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39961 (local address 127.27.115.193:45825)
0811 18:50:18.569971 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:18.569987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:18.570274 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:18.570588 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:18.570591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:18.570599 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:50:18.570715 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.570721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.571566 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.571570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.572303 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:18.572313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:18.572880 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:18.572883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:18.572975 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:18.573013 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:50:18.573367 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:50:18.573374 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:18.573424 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:50:18.573509 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":139,"threads_started":1}
I20260811 18:50:18.599499 4408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.599689 4408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:18.602391 4408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc34075ef0ca494b9acb05371f001134: Sending DeleteTablet for 3 replicas of tablet fa2eec9b23ea4afab9f6c356da54fecd
I20260811 18:50:18.603214 4896 tablet_service.cc:1514] Processing DeleteTablet for tablet fa2eec9b23ea4afab9f6c356da54fecd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:59198
I20260811 18:50:18.603236 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet fa2eec9b23ea4afab9f6c356da54fecd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:18 UTC) from {username='oryx', principal='oryx/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34326
I20260811 18:50:18.603436 5083 tablet_replica.cc:330] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: stopping tablet replica
I20260811 18:50:18.603472 5082 tablet_replica.cc:330] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: stopping tablet replica
I20260811 18:50:18.603559 5083 raft_consensus.cc:2229] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:18.603590 5082 raft_consensus.cc:2229] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:18.603713 5082 raft_consensus.cc:2258] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.603947 5083 raft_consensus.cc:2258] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:18.603996 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 4489
I20260811 18:50:18.607309 5083 ts_tablet_manager.cc:1905] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.608856 5082 ts_tablet_manager.cc:1905] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:18.609812 5083 ts_tablet_manager.cc:1918] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:50:18.609920 5083 log.cc:1196] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/fa2eec9b23ea4afab9f6c356da54fecd
I20260811 18:50:18.610220 5083 ts_tablet_manager.cc:1939] T fa2eec9b23ea4afab9f6c356da54fecd P 1e2f848977314ed5984a6c1aaf4bcae9: Deleting consensus metadata
I20260811 18:50:18.611056 4395 catalog_manager.cc:4841] TS 1e2f848977314ed5984a6c1aaf4bcae9 (127.27.115.195:39961): tablet fa2eec9b23ea4afab9f6c356da54fecd (table test-table [id=c5fe8cc00ada46a29a10bdaeabffe4da]) successfully deleted
I20260811 18:50:18.612249 5082 ts_tablet_manager.cc:1918] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:50:18.612346 5082 log.cc:1196] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/fa2eec9b23ea4afab9f6c356da54fecd
I20260811 18:50:18.612651 5082 ts_tablet_manager.cc:1939] T fa2eec9b23ea4afab9f6c356da54fecd P cb534b7494b4478e905024d64e934048: Deleting consensus metadata
I20260811 18:50:18.613559 4393 catalog_manager.cc:4841] TS cb534b7494b4478e905024d64e934048 (127.27.115.194:42871): tablet fa2eec9b23ea4afab9f6c356da54fecd (table test-table [id=c5fe8cc00ada46a29a10bdaeabffe4da]) successfully deleted
W20260811 18:50:18.618023 4395 connection.cc:535] client connection to 127.27.115.193:41109 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:50:18.618276 4395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:50:18.618561 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 4654
I20260811 18:50:18.622181 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:18.621581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41109 (local address 127.0.0.1:36694)
0811 18:50:18.621923 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:18.622018 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
W20260811 18:50:18.622623 4395 catalog_manager.cc:4584] TS 36f47d8b920e40918fc10aa454a9312e (127.27.115.193:41109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa2eec9b23ea4afab9f6c356da54fecd: Network error: Client connection negotiation failed: client connection to 127.27.115.193:41109: connect: Connection refused (error 111)
I20260811 18:50:18.630240 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 4819
I20260811 18:50:18.640538 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 4374
2026-08-11T18:50:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4193 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[5139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[5139](info): set up 2 sockets
Aug 11 18:50:18 dist-test-slave-9jjr krb5kdc[5139](info): commencing operation
krb5kdc: starting...
W20260811 18:50:19.774053 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:19 dist-test-slave-9jjr krb5kdc[5139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:50:19.913332 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:38681
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:36667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:38681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:20.054059 5154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:20.054469 5154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:20.054553 5154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:20.059017 5154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:20.059157 5154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:20.059195 5154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:20.059229 5154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:20.059284 5154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:20.065455 5154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:36667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:38681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:38681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:20.066825 5154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:20.067998 5154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:20.078212 5163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:20.078212 5159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:20.078212 5160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:20.078663 5154 server_base.cc:1034] running on GCE node
I20260811 18:50:20.079361 5154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:20.080617 5154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:20.081804 5154 hybrid_clock.cc:648] HybridClock initialized: now 1786474220081774 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T18:50:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1446 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:20 dist-test-slave-9jjr krb5kdc[5168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:20 dist-test-slave-9jjr krb5kdc[5168](info): set up 2 sockets
Aug 11 18:50:20 dist-test-slave-9jjr krb5kdc[5168](info): commencing operation
krb5kdc: starting...
W20260811 18:50:21.212090 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:21 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:50:21.350347 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37875
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:21.477917 5182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:21.478386 5182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:21.478475 5182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:21.482744 5182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:21.482863 5182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:21.482899 5182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:21.482944 5182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:21.482976 5182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:21.488865 5182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:21.490240 5182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:21.491446 5182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:21.501788 5189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:21.501776 5191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:21.501761 5188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:21.503012 5182 server_base.cc:1034] running on GCE node
I20260811 18:50:21.503990 5182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:21.505591 5182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:21.506835 5182 hybrid_clock.cc:648] HybridClock initialized: now 1786474221506805 us; error 51 us; skew 500 ppm
Aug 11 18:50:21 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:21.511401 5182 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:50:21.513070 5182 webserver.cc:460] Webserver started at http://127.27.115.254:37133/ using document root <none> and password file <none>
I20260811 18:50:21.518457 5182 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:21.518721 5182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:21.519079 5182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:21.522763 5182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "9d8161e85be24dbcb4697932f27d2fdf"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "269d426b8a6c5155bd23c0d5d442d966"
server_key_iv: "748a425de1078a200bdfd35118d8a188"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.523751 5182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "9d8161e85be24dbcb4697932f27d2fdf"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "269d426b8a6c5155bd23c0d5d442d966"
server_key_iv: "748a425de1078a200bdfd35118d8a188"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.529838 5182 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 18:50:21.533229 5200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:21.534950 5182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:21.535132 5182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "9d8161e85be24dbcb4697932f27d2fdf"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "269d426b8a6c5155bd23c0d5d442d966"
server_key_iv: "748a425de1078a200bdfd35118d8a188"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.535315 5182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:21.567606 5182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:21.571764 5182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:21.571985 5182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:21.582967 5182 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37875
I20260811 18:50:21.582973 5258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37875 every 8 connection(s)
I20260811 18:50:21.584393 5182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:21.588181 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:21.591259 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5182
I20260811 18:50:21.591418 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:21.591722 28111 external_mini_cluster.cc:1428] Setting key 0cb76841a0467b7f9709eafffe68f34c
I20260811 18:50:21.598353 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Bootstrap starting.
I20260811 18:50:21.601759 5259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:21.602993 5259 log.cc:824] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:50:21 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:21.605978 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: No bootstrap required, opened a new log
I20260811 18:50:21.610481 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:21.593919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.27.115.254:37875)
0811 18:50:21.594505 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:21.594529 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:21.594577 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:50:21.595709 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:21.595740 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:21.596234 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:21.597693 (+ 1459us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:21.598050 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.599302 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.600323 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.601147 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.601844 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:21.605186 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:21.605194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:21.605213 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:21.605247 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:21.607457 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:21.607949 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:21.607955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:21.607960 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:21.608074 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:21.608371 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:21.608375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:21.608377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:21.608808 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:21.609000 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:21.609310 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 18:50:21.609613 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":216,"threads_started":1}
I20260811 18:50:21.611404 5259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:21.611925 5259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:21.611995 5259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d8161e85be24dbcb4697932f27d2fdf, State: Initialized, Role: FOLLOWER
I20260811 18:50:21.612475 5259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:21.612644 5259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:21.612711 5259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:21.612811 5259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:21.614456 5259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:21.615068 5259 leader_election.cc:302] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d8161e85be24dbcb4697932f27d2fdf; no voters:
I20260811 18:50:21.615460 5259 leader_election.cc:288] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:21.615609 5264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:21.615899 5264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 LEADER]: Becoming Leader. State: Replica: 9d8161e85be24dbcb4697932f27d2fdf, State: Running, Role: LEADER
I20260811 18:50:21.616220 5264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:21.617009 5259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:21.617993 5266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d8161e85be24dbcb4697932f27d2fdf. Latest consensus state: current_term: 1 leader_uuid: "9d8161e85be24dbcb4697932f27d2fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } } }
I20260811 18:50:21.618067 5265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d8161e85be24dbcb4697932f27d2fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } } }
I20260811 18:50:21.618127 5266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:21.618286 5265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:21.622603 5280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:21.622709 5280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:21.622802 5274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:21.623569 5274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:21.629781 5274 catalog_manager.cc:1347] Generated new cluster ID: 80af8a055a864d2b97e178fbd2130ff2
I20260811 18:50:21.629877 5274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:21.652292 5274 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:21.653522 5274 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:21.663833 5274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Generated new TSK 0
I20260811 18:50:21.664724 5274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:50:21.707059 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:21.836658 5286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:21.837150 5286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:21.837248 5286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:21.842306 5286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:21.842429 5286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:21.842535 5286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:21.848930 5286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:21.850507 5286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:21.851922 5286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:21.863514 5292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:21.863780 5286 server_base.cc:1034] running on GCE node
W20260811 18:50:21.863544 5296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:21.863544 5293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:21.864442 5286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:21.865229 5286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:21.866478 5286 hybrid_clock.cc:648] HybridClock initialized: now 1786474221866455 us; error 38 us; skew 500 ppm
Aug 11 18:50:21 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:21.869668 5286 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:50:21.870728 5286 webserver.cc:460] Webserver started at http://127.27.115.193:43363/ using document root <none> and password file <none>
I20260811 18:50:21.871315 5286 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:21.871376 5286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:21.871559 5286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:21.873787 5286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "b13c62331d3f405990111cb495aa8a5f"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "9de66fdbe4d2295aca0143370c814797"
server_key_iv: "603c8cb16ca863546abaf88d615b7bfc"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.874420 5286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "b13c62331d3f405990111cb495aa8a5f"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "9de66fdbe4d2295aca0143370c814797"
server_key_iv: "603c8cb16ca863546abaf88d615b7bfc"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.878680 5286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:50:21.881496 5308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:21.883028 5286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:21.883178 5286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "b13c62331d3f405990111cb495aa8a5f"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "9de66fdbe4d2295aca0143370c814797"
server_key_iv: "603c8cb16ca863546abaf88d615b7bfc"
server_key_version: "encryptionkey@0"
I20260811 18:50:21.883289 5286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:21.899950 5286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:21.903858 5286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:21.904048 5286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:21.904671 5286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:21.905627 5286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:21.905680 5286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:21.905727 5286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:21.905750 5286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:21.915795 5286 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:33587
I20260811 18:50:21.915885 5432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:33587 every 8 connection(s)
I20260811 18:50:21.916987 5286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:21.926115 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5286
I20260811 18:50:21.926318 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:21.926651 28111 external_mini_cluster.cc:1428] Setting key b7cc45f1cef80370e02b691d26ab6dbd
Aug 11 18:50:21 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:21.933933 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:21.919580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:46753 (local address 127.27.115.254:37875)
0811 18:50:21.919765 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:21.919770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:21.920993 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 18:50:21.922538 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:21.922541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:21.922598 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:21.922683 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:21.924170 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.924777 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.926238 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.926568 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.926673 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:21.929712 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:21.929720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:21.929726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:21.929766 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:21.931509 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:21.932253 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:21.932256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:21.932257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:21.932343 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:21.932730 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:21.932733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:21.932735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:21.932942 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:21.933049 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:21.933607 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 18:50:21.933759 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:50:21.935070 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:21.920041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:46753)
0811 18:50:21.920681 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:21.920755 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:21.922298 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:21.922924 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:21.922933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:21.923381 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:21.923922 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:21.923937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.924921 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.924924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:21.926037 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:21.926052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:21.926766 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:21.926772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:21.926982 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:21.927730 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:21.927747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:21.929517 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:21.931658 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:21.931666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:21.931708 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:21.932107 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:21.932454 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:21.932457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:21.932459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:21.932613 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:21.933065 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:21.933074 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:21.933359 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 18:50:21.933565 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 18:50:21.933945 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":178,"threads_started":1}
I20260811 18:50:21.936520 5434 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37875
I20260811 18:50:21.936813 5434 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:21.937737 5434 heartbeater.cc:511] Master 127.27.115.254:37875 requested a full tablet report, sending...
I20260811 18:50:21.939553 5220 ts_manager.cc:194] Registered new tserver with Master: b13c62331d3f405990111cb495aa8a5f (127.27.115.193:33587)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:50:21.941239 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:46753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:50:21.962440 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:22.087024 5439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:22.087393 5439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:22.087445 5439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:22.092237 5439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:22.092334 5439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:22.092402 5439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:22.098770 5439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:22.100095 5439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:22.101534 5439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:22.113574 5445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:22.113713 5439 server_base.cc:1034] running on GCE node
W20260811 18:50:22.113566 5448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:22.113566 5446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:22.114540 5439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:22.115401 5439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:22.116684 5439 hybrid_clock.cc:648] HybridClock initialized: now 1786474222116642 us; error 72 us; skew 500 ppm
Aug 11 18:50:22 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:22.120735 5439 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:50:22.122108 5439 webserver.cc:460] Webserver started at http://127.27.115.194:46155/ using document root <none> and password file <none>
I20260811 18:50:22.122885 5439 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:22.122999 5439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:22.123315 5439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:22.125872 5439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "59d84c8b647d43adb10c95929b3c38df"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "dfcb0a2b677724b510290938c4385bcf"
server_key_iv: "ba331fb68fccfce12e23fe14446e1031"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.126703 5439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "59d84c8b647d43adb10c95929b3c38df"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "dfcb0a2b677724b510290938c4385bcf"
server_key_iv: "ba331fb68fccfce12e23fe14446e1031"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.131712 5439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:50:22.134936 5457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.136602 5439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:22.136813 5439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "59d84c8b647d43adb10c95929b3c38df"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "dfcb0a2b677724b510290938c4385bcf"
server_key_iv: "ba331fb68fccfce12e23fe14446e1031"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.136973 5439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:22.153970 5439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:22.157866 5439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:22.158164 5439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:22.158866 5439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:22.159999 5439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:22.160089 5439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.160180 5439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:22.160235 5439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.170522 5439 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:35147
I20260811 18:50:22.170542 5594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:35147 every 8 connection(s)
I20260811 18:50:22.171617 5439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:50:22 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:22.181954 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5439
I20260811 18:50:22.182096 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:22.182428 28111 external_mini_cluster.cc:1428] Setting key f5e120014d5d0e9f3a032312ee1271e5
I20260811 18:50:22.186470 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:22.173560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:50785 (local address 127.27.115.254:37875)
0811 18:50:22.173698 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:22.173702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:22.174693 (+ 991us) server_negotiation.cc:407] Connection header received
0811 18:50:22.176097 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:22.176101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:22.176162 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:22.176247 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:22.177497 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.178042 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.179278 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.179504 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.179588 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:22.182519 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:22.182525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:22.182527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:22.182560 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:22.184150 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:22.184744 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:22.184747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:22.184749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:22.184828 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:22.185214 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:22.185218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:22.185220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:22.185426 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:22.185537 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:22.186119 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 18:50:22.186298 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:50:22.187481 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:22.173916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:50785)
0811 18:50:22.174522 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:22.174572 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:22.175870 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:22.176414 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:22.176421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:22.176819 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:22.177351 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:22.177366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.178218 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.178220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:22.179158 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:22.179169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.179648 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.179651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:22.179816 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:22.180540 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:22.180558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:22.182292 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:22.184284 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:22.184292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:22.184311 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:22.184623 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:22.184923 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:22.184927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:22.184929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:22.185108 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:22.185538 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:22.185547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:22.185848 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 18:50:22.186061 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 18:50:22.186483 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":240,"threads_started":1}
I20260811 18:50:22.188807 5595 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37875
I20260811 18:50:22.189071 5595 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:22.189620 5595 heartbeater.cc:511] Master 127.27.115.254:37875 requested a full tablet report, sending...
I20260811 18:50:22.190835 5220 ts_manager.cc:194] Registered new tserver with Master: 59d84c8b647d43adb10c95929b3c38df (127.27.115.194:35147)
I20260811 18:50:22.191450 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:50785
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:50:22.214672 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:22.345571 5601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:22.345953 5601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:22.346004 5601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:22.350867 5601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:22.350973 5601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:22.351042 5601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:22.357381 5601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:22.358793 5601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:22.359956 5601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:22.371155 5610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:22.371335 5601 server_base.cc:1034] running on GCE node
W20260811 18:50:22.371130 5607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:22.371224 5606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:22.371982 5601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:22.372717 5601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:22.373898 5601 hybrid_clock.cc:648] HybridClock initialized: now 1786474222373873 us; error 40 us; skew 500 ppm
Aug 11 18:50:22 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:22.377343 5601 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:50:22.378628 5601 webserver.cc:460] Webserver started at http://127.27.115.195:41487/ using document root <none> and password file <none>
I20260811 18:50:22.379305 5601 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:22.379381 5601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:22.379627 5601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:22.382429 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "b5f98543385d4f6f80270692fddbbc7b"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "ce81f558bfee9072c6b595433f205d37"
server_key_iv: "ecea932a9313658825ce3a5693ff9db7"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.383217 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "b5f98543385d4f6f80270692fddbbc7b"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "ce81f558bfee9072c6b595433f205d37"
server_key_iv: "ecea932a9313658825ce3a5693ff9db7"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.387894 5601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:22.391026 5617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.392596 5601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:22.392810 5601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "b5f98543385d4f6f80270692fddbbc7b"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "ce81f558bfee9072c6b595433f205d37"
server_key_iv: "ecea932a9313658825ce3a5693ff9db7"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.392968 5601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:22.413486 5601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:22.417272 5601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:22.417508 5601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:22.418268 5601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:22.419245 5601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:22.419318 5601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.419396 5601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:22.419438 5601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.429863 5601 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:36665
I20260811 18:50:22.429906 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:36665 every 8 connection(s)
I20260811 18:50:22.431030 5601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:22.434450 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5601
I20260811 18:50:22.434665 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:22.435066 28111 external_mini_cluster.cc:1428] Setting key e4abdf7295c4ba58ec9fbf69150a771d
Aug 11 18:50:22 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:22.449745 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:22.433404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:37219 (local address 127.27.115.254:37875)
0811 18:50:22.433586 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:22.433591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:22.435445 (+ 1854us) server_negotiation.cc:407] Connection header received
0811 18:50:22.436809 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:22.436812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:22.436875 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:22.436971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:22.438564 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.439145 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.440777 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.441069 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.441222 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:22.445067 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:22.445071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:22.445073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:22.445118 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:22.446870 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:22.447581 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:22.447584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:22.447585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:22.447678 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:22.448191 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:22.448215 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:22.448217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:22.448479 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:22.448599 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:22.449385 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 18:50:22.449535 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:50:22.450784 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:22.433818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:37219)
0811 18:50:22.435210 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:22.435295 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:22.436503 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:22.437170 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:22.437178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:22.437738 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:22.438379 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:22.438400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.439330 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.439333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:22.440580 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:22.440596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:22.441334 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:22.441339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:22.441624 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:22.442702 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:22.442726 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:22.444784 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:22.447053 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:22.447061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:22.447079 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:22.447448 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:22.447816 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:22.447819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:22.447821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:22.448057 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:22.448607 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:22.448614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:22.449029 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 18:50:22.449292 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 18:50:22.449728 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"thread_start_us":924,"threads_started":1}
I20260811 18:50:22.452163 5746 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37875
I20260811 18:50:22.452476 5746 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:22.453106 5746 heartbeater.cc:511] Master 127.27.115.254:37875 requested a full tablet report, sending...
I20260811 18:50:22.454346 5220 ts_manager.cc:194] Registered new tserver with Master: b5f98543385d4f6f80270692fddbbc7b (127.27.115.195:36665)
I20260811 18:50:22.454938 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:37219
I20260811 18:50:22.462112 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
Entry for principal oryx/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab.
WARNING: no policy specified for oryx/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
Entry for principal oryx/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab.
WARNING: no policy specified for oryx/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
Entry for principal oryx/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab.
I20260811 18:50:22.557765 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5286
I20260811 18:50:22.566203 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5439
I20260811 18:50:22.575577 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5601
I20260811 18:50:22.584453 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5182
I20260811 18:50:22.593940 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37875
--webserver_interface=127.27.115.254
--webserver_port=37133
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:22.717814 5756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:22.718231 5756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:22.718345 5756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:22.722724 5756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:22.722824 5756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:22.722862 5756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:22.722903 5756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:22.722937 5756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:22.728778 5756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=37133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:22.730101 5756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:22.731312 5756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:22.741411 5762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:22.741411 5764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:22.741458 5761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:22.741916 5756 server_base.cc:1034] running on GCE node
I20260811 18:50:22.742805 5756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:22.744175 5756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:22.745375 5756 hybrid_clock.cc:648] HybridClock initialized: now 1786474222745347 us; error 54 us; skew 500 ppm
Aug 11 18:50:22 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:22.748453 5756 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:50:22.749524 5756 webserver.cc:460] Webserver started at http://127.27.115.254:37133/ using document root <none> and password file <none>
I20260811 18:50:22.750207 5756 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:22.750322 5756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:22.754834 5756 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:50:22.757315 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:22.758785 5756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:22.758945 5756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "9d8161e85be24dbcb4697932f27d2fdf"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "269d426b8a6c5155bd23c0d5d442d966"
server_key_iv: "748a425de1078a200bdfd35118d8a188"
server_key_version: "encryptionkey@0"
I20260811 18:50:22.759558 5756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:22.775167 5756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:22.779920 5756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:22.780171 5756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:22.788305 5756 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37875
I20260811 18:50:22.788362 5837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37875 every 8 connection(s)
I20260811 18:50:22.789567 5756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:22.792868 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5756
I20260811 18:50:22.793432 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:33587
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:37875
--webserver_port=43363
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:22.795027 5838 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:22.797171 5838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Bootstrap starting.
I20260811 18:50:22.824023 5838 log.cc:824] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:22.833833 5838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:22.834410 5838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Bootstrap complete.
I20260811 18:50:22.840798 5838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:22.841465 5838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d8161e85be24dbcb4697932f27d2fdf, State: Initialized, Role: FOLLOWER
I20260811 18:50:22.842180 5838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:22.842342 5838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:22.842418 5838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:22.842520 5838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:50:22.845414 5838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:22.845821 5838 leader_election.cc:302] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d8161e85be24dbcb4697932f27d2fdf; no voters:
I20260811 18:50:22.846246 5838 leader_election.cc:288] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:50:22.846437 5843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:50:22.846742 5843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [term 2 LEADER]: Becoming Leader. State: Replica: 9d8161e85be24dbcb4697932f27d2fdf, State: Running, Role: LEADER
I20260811 18:50:22.847088 5843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } }
I20260811 18:50:22.847658 5838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:22.848732 5845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d8161e85be24dbcb4697932f27d2fdf. Latest consensus state: current_term: 2 leader_uuid: "9d8161e85be24dbcb4697932f27d2fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } } }
I20260811 18:50:22.848908 5845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:22.848732 5844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d8161e85be24dbcb4697932f27d2fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d8161e85be24dbcb4697932f27d2fdf" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37875 } } }
I20260811 18:50:22.849066 5844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:22.849496 5855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:22.852643 5855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:22.853268 5855 catalog_manager.cc:1259] Loaded cluster ID: 80af8a055a864d2b97e178fbd2130ff2
I20260811 18:50:22.853348 5855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:22.855185 5855 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:22.855844 5855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d8161e85be24dbcb4697932f27d2fdf: Loaded TSK: 0
I20260811 18:50:22.856866 5855 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:50:22.976444 5840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:22.976879 5840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:22.976948 5840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:22.983345 5840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:22.983449 5840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:22.983563 5840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:22.991637 5840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.193.keytab
--principal=oryx/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:33587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=43363
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:22.993011 5840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:22.994403 5840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:23.006488 5874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.006512 5871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.006538 5870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:23.006685 5840 server_base.cc:1034] running on GCE node
I20260811 18:50:23.007355 5840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:23.008141 5840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:23.009344 5840 hybrid_clock.cc:648] HybridClock initialized: now 1786474223009316 us; error 47 us; skew 500 ppm
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474223, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:23.012549 5840 init.cc:377] Logged in from keytab as oryx/127.27.115.193@KRBTEST.COM (short username oryx)
I20260811 18:50:23.013662 5840 webserver.cc:460] Webserver started at http://127.27.115.193:43363/ using document root <none> and password file <none>
I20260811 18:50:23.014322 5840 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:23.014425 5840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:23.018692 5840 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 18:50:23.021052 5882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.022636 5840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:23.022787 5840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "b13c62331d3f405990111cb495aa8a5f"
format_stamp: "Formatted at 2026-08-11 18:50:21 on dist-test-slave-9jjr"
server_key: "9de66fdbe4d2295aca0143370c814797"
server_key_iv: "603c8cb16ca863546abaf88d615b7bfc"
server_key_version: "encryptionkey@0"
I20260811 18:50:23.023545 5840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:23.039269 5840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:23.043211 5840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:23.043459 5840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:23.044240 5840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:23.045194 5840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:23.045267 5840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.045336 5840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:23.045380 5840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.057665 5840 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:33587
I20260811 18:50:23.057700 6014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:33587 every 8 connection(s)
I20260811 18:50:23.058804 5840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:23.064200 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 5840
I20260811 18:50:23.064760 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:35147
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:37875
--webserver_port=46155
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:50:23.073583 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.060995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:33947 (local address 127.27.115.254:37875)
0811 18:50:23.061517 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.061537 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.061784 (+ 247us) server_negotiation.cc:407] Connection header received
0811 18:50:23.063388 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.063400 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.063812 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.064219 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.065468 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.066515 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.067773 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.068250 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.068426 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.072842 (+ 4416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:33947: BlockingRecv error: recv got EOF from 127.27.115.193:33947 (error 108)
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":104,"threads_started":1}
I20260811 18:50:23.073711 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.061054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:33947)
0811 18:50:23.061626 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.061668 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.062966 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.064256 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.064264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.064753 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.065289 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.065304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.066667 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.066670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.067642 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.067658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.068429 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.068433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.068630 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.069315 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.069332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.072592 (+ 3260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":165,"threads_started":1}
W20260811 18:50:23.074239 6015 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37875 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.115.254:37875: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.080637 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.074702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:44005)
0811 18:50:23.074877 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.074894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.074996 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.075265 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.075270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.075476 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.075630 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.075654 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.076399 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.076404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.077449 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.077466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.077891 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.077894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.077995 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.078504 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.078518 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.080481 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 18:50:23.080861 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.074829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:44005 (local address 127.27.115.254:37875)
0811 18:50:23.074934 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.074938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.074951 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:23.075099 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.075101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.075144 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.075213 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.075770 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.076287 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.077577 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.077786 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.077858 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.080611 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:44005: BlockingRecv error: recv got EOF from 127.27.115.193:44005 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.086828 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.081203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:47909)
0811 18:50:23.081388 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.081409 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.081469 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.081699 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.081701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.081835 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.081909 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.081914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.082936 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.082939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.083477 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.083487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.084138 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.084141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.084206 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.084723 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.084736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.086720 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118}
W20260811 18:50:23.087003 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.081314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:47909 (local address 127.27.115.254:37875)
0811 18:50:23.081443 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.081447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.081462 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:50:23.081549 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.081552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.081594 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.081676 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.082005 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.082823 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.083592 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.083992 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.084126 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.086888 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:47909: BlockingRecv error: recv got EOF from 127.27.115.193:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:50:23.087168 6015 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:50:23.188658 6021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:23.189013 6021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:23.189069 6021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:23.193331 6021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:23.193413 6021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:23.193475 6021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:23.199275 6021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.194.keytab
--principal=oryx/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:35147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=46155
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:23.200469 6021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:23.201529 6021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:23.213349 6027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.213351 6030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.213351 6028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:23.214301 6021 server_base.cc:1034] running on GCE node
I20260811 18:50:23.214835 6021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:23.215852 6021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:23.217067 6021 hybrid_clock.cc:648] HybridClock initialized: now 1786474223217050 us; error 38 us; skew 500 ppm
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474223, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:23.220415 6021 init.cc:377] Logged in from keytab as oryx/127.27.115.194@KRBTEST.COM (short username oryx)
I20260811 18:50:23.221546 6021 webserver.cc:460] Webserver started at http://127.27.115.194:46155/ using document root <none> and password file <none>
I20260811 18:50:23.222368 6021 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:23.222489 6021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:23.228094 6021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:50:23.231364 6038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.233192 6021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:23.233400 6021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "59d84c8b647d43adb10c95929b3c38df"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "dfcb0a2b677724b510290938c4385bcf"
server_key_iv: "ba331fb68fccfce12e23fe14446e1031"
server_key_version: "encryptionkey@0"
I20260811 18:50:23.234087 6021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:23.257568 6021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:23.260854 6021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:23.261076 6021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:23.261734 6021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:23.262744 6021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:23.262804 6021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.262871 6021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:23.262915 6021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.273221 6021 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:35147
I20260811 18:50:23.273232 6176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:35147 every 8 connection(s)
I20260811 18:50:23.274466 6021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:23.283340 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6021
I20260811 18:50:23.283861 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:36665
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:37875
--webserver_port=41487
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:50:23.287329 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.276508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:52323 (local address 127.27.115.254:37875)
0811 18:50:23.276644 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.276648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.277566 (+ 918us) server_negotiation.cc:407] Connection header received
0811 18:50:23.279103 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.279106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.279176 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.279286 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.280612 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.281153 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.282449 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.282703 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.282820 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.287158 (+ 4338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:52323: BlockingRecv error: recv got EOF from 127.27.115.194:52323 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:50:23.287889 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.276883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:52323)
0811 18:50:23.277396 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.277446 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.278863 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.279485 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.279493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.279906 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.280437 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.280452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.281300 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.281303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.282318 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.282330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.282875 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.282879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.283054 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.283815 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.283832 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.286899 (+ 3067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":142,"threads_started":1}
W20260811 18:50:23.288627 6177 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37875 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.115.254:37875: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.296140 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.289305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:37689)
0811 18:50:23.289441 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.289457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.289565 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.289831 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.289834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.290040 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.290308 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.290322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.291186 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.291190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.291796 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.291806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.292310 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.292314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.292420 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.293133 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.293146 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.295962 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:50:23.296872 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.289432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37689 (local address 127.27.115.254:37875)
0811 18:50:23.289583 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.289590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.289612 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:50:23.289665 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.289668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.289719 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.289809 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.290473 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.291048 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.291936 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.292172 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.292296 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.296753 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:37689: BlockingRecv error: recv got EOF from 127.27.115.194:37689 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.304519 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.298030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:42189)
0811 18:50:23.298268 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.298288 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.298416 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.298770 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.298773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.298983 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.299080 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.299086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.300020 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.300025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.300604 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.300613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.301131 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.301134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.301220 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.301869 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.301882 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.304353 (+ 2471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 18:50:23.304708 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.298348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:42189 (local address 127.27.115.254:37875)
0811 18:50:23.298528 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.298534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.298555 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:50:23.298601 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.298604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.298645 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.298743 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.299224 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.299895 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.300755 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.301001 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.301123 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.304534 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:42189: BlockingRecv error: recv got EOF from 127.27.115.194:42189 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 18:50:23.305052 6177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:50:23.413468 6180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:23.413870 6180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:23.413955 6180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:23.419412 6180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:23.419553 6180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:23.419661 6180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:23.426203 6180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/krb5kdc/oryx_127.27.115.195.keytab
--principal=oryx/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:36665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=41487
--tserver_master_addrs=127.27.115.254:37875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:23.427702 6180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:23.429085 6180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:23.440876 6186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.440943 6187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:23.441010 6189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:23.441262 6180 server_base.cc:1034] running on GCE node
I20260811 18:50:23.441798 6180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:23.442682 6180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:23.443910 6180 hybrid_clock.cc:648] HybridClock initialized: now 1786474223443867 us; error 61 us; skew 500 ppm
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474223, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:23.447587 6180 init.cc:377] Logged in from keytab as oryx/127.27.115.195@KRBTEST.COM (short username oryx)
I20260811 18:50:23.449131 6180 webserver.cc:460] Webserver started at http://127.27.115.195:41487/ using document root <none> and password file <none>
I20260811 18:50:23.449927 6180 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:23.450071 6180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:23.455147 6180 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:50:23.457799 6196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.459389 6180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:50:23.459530 6180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "b5f98543385d4f6f80270692fddbbc7b"
format_stamp: "Formatted at 2026-08-11 18:50:22 on dist-test-slave-9jjr"
server_key: "ce81f558bfee9072c6b595433f205d37"
server_key_iv: "ecea932a9313658825ce3a5693ff9db7"
server_key_version: "encryptionkey@0"
I20260811 18:50:23.460263 6180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:23.474735 6180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:23.478263 6180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:23.478495 6180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:23.479187 6180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:23.480080 6180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:23.480139 6180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.480211 6180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:23.480250 6180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:23.490255 6180 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:36665
I20260811 18:50:23.490342 6328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:36665 every 8 connection(s)
I20260811 18:50:23.491590 6180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:23.493602 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6180
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.504691 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.494799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.27.115.254:37875)
0811 18:50:23.495020 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.495023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.495035 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:50:23.495078 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.495080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.495118 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.495270 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.496067 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.496648 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.497856 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.498085 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.498332 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.500047 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:23.500054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:23.500068 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:23.500103 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:23.502115 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:23.502719 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:23.502725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:23.502731 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:23.502921 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:23.503238 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:23.503244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:23.503246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:23.503748 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:23.503982 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:23.504255 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 18:50:23.504516 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
W20260811 18:50:23.505199 6019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.493875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:48585 (local address 127.27.115.254:37875)
0811 18:50:23.494035 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.494042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.495204 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 18:50:23.496596 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.496599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.496648 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.496732 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.498261 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.499020 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.500695 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.501067 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.501197 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.505096 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:48585: BlockingRecv error: recv got EOF from 127.27.115.195:48585 (error 108)
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":50}
I20260811 18:50:23.505908 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.494484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:48585)
0811 18:50:23.495034 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.495088 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.496403 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.496942 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.496951 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.497446 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.498075 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.498091 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.499167 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.499171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.500504 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.500517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.501267 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.501272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.501475 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.502029 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.502045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.504988 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":164,"threads_started":1}
W20260811 18:50:23.506536 6329 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:37875 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.115.254:37875: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.512488 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.507055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:55571)
0811 18:50:23.507202 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.507218 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.507310 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.507546 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.507549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.507779 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.507906 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.507913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.508660 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.508663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.509226 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.509234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.509716 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.509720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.509818 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.510419 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.510431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.512322 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:50:23.512596 6335 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.507146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:55571 (local address 127.27.115.254:37875)
0811 18:50:23.507277 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.507282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.507296 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:23.507398 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.507401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.507440 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.507527 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.508023 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.508545 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.509355 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.509593 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.509704 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.512479 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:55571: BlockingRecv error: recv got EOF from 127.27.115.195:55571 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:23 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:23.518460 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:23.513114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:42475)
0811 18:50:23.513260 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:23.513276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:23.513386 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:23.513640 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:23.513643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:23.513815 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:23.513893 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.513898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.514752 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.514756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.515280 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:23.515289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.515742 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.515746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:23.515827 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.516353 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:23.516364 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:23.518287 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 18:50:23.518556 6335 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:23.513234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:42475 (local address 127.27.115.254:37875)
0811 18:50:23.513416 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:23.513423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:23.513452 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:50:23.513500 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:23.513502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:23.513538 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:23.513622 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:23.514007 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.514639 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.515403 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:23.515628 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:23.515737 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:23.518427 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:42475: BlockingRecv error: recv got EOF from 127.27.115.195:42475 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 18:50:23.519058 6329 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:24.095294 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:24.088065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:34751)
0811 18:50:24.088463 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:24.088481 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:24.088636 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:24.089129 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:24.089132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:24.089382 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:24.089521 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.089528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.090532 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.090535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.091225 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.091238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.091873 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.091877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.091979 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.092652 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:24.092666 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:24.095135 (+ 2469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":100,"threads_started":1}
W20260811 18:50:24.095714 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:24.088211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:34751 (local address 127.27.115.254:37875)
0811 18:50:24.088529 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:24.088534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:24.088553 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:50:24.088868 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:24.088871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:24.088932 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:24.089031 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:24.089681 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.090404 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.091388 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.091686 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.091815 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.095549 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:34751: BlockingRecv error: recv got EOF from 127.27.115.193:34751 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":132,"threads_started":1}
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:24.311902 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:24.305813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:37969)
0811 18:50:24.306207 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:24.306225 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:24.306350 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:24.306656 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:24.306659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:24.306854 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:24.306960 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.306965 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.307814 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.307817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.308457 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.308467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.308936 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.308938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.309004 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.309639 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:24.309652 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:24.311691 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":110,"threads_started":1}
W20260811 18:50:24.311951 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:24.305866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37969 (local address 127.27.115.254:37875)
0811 18:50:24.306037 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:24.306042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:24.306308 (+ 266us) server_negotiation.cc:407] Connection header received
0811 18:50:24.306437 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:24.306439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:24.306490 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:24.306562 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:24.307085 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.307680 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.308584 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.308815 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.308900 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.311842 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:37969: BlockingRecv error: recv got EOF from 127.27.115.194:37969 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:24 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:24.526378 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:24.520312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:47523)
0811 18:50:24.520636 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:24.520659 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:24.520783 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:24.521020 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:24.521022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:24.521238 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:24.521338 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.521344 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.522260 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.522263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.522857 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:24.522867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.523377 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.523380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:24.523446 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.524147 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:24.524166 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:24.526243 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":147,"threads_started":1}
W20260811 18:50:24.526563 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:24.520539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:47523 (local address 127.27.115.254:37875)
0811 18:50:24.520724 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:24.520727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:24.520745 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:50:24.520857 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:24.520859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:24.520906 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:24.520989 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:24.521491 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.522068 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.522997 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:24.523248 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:24.523368 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:24.526435 (+ 3067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:47523: BlockingRecv error: recv got EOF from 127.27.115.195:47523 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:25.110541 6340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:25.096272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:42347)
0811 18:50:25.096599 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:25.096612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:25.096719 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:25.096982 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:25.096984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:25.097190 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:25.097298 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.097304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.098115 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.098119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.098803 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.098813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.099738 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.099742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.099835 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.108208 (+ 8373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:25.108226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:25.110379 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
W20260811 18:50:25.110747 6341 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:25.096405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:42347 (local address 127.27.115.254:37875)
0811 18:50:25.096680 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:25.096683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:25.096697 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:25.096811 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:25.096814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:25.096860 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:25.096942 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:25.097438 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.097986 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.099135 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.099417 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.099526 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.110587 (+ 11061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:42347: BlockingRecv error: recv got EOF from 127.27.115.193:42347 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:25.318331 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:25.312892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:43859)
0811 18:50:25.313192 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:25.313205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:25.313314 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:25.313533 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:25.313535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:25.313722 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:25.313824 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.313830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.314627 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.314630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.315190 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.315199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.315600 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.315603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.315664 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.316230 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:25.316250 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:25.318160 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
W20260811 18:50:25.318593 6341 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:25.312934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:43859 (local address 127.27.115.254:37875)
0811 18:50:25.313093 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:25.313098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:25.313281 (+ 183us) server_negotiation.cc:407] Connection header received
0811 18:50:25.313382 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:25.313384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:25.313430 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:25.313499 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:25.313924 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.314507 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.315295 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.315504 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.315578 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.318465 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:43859: BlockingRecv error: recv got EOF from 127.27.115.194:43859 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:25 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:25.535104 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:25.527595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:53947)
0811 18:50:25.528042 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:25.528063 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:25.528280 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:25.528682 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:25.528684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:25.528897 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:25.529218 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.529262 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.530416 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.530420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.531293 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:25.531309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.531923 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.531929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:25.532010 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.532743 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:25.532757 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:25.534967 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":137,"threads_started":1}
W20260811 18:50:25.535389 6341 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:25.527880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:53947 (local address 127.27.115.254:37875)
0811 18:50:25.528042 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:25.528049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:25.528158 (+ 109us) server_negotiation.cc:407] Connection header received
0811 18:50:25.528474 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:25.528477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:25.528537 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:25.528628 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:25.529530 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.530183 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.531434 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:25.531755 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:25.531869 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:25.535243 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:53947: BlockingRecv error: recv got EOF from 127.27.115.195:53947 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:26.117501 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:26.111719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:48617)
0811 18:50:26.112076 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:26.112095 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:26.112251 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:26.112606 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:26.112610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:26.112827 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:26.112938 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.112945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.113707 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.113710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.114354 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.114363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.114810 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.114812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.114875 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.115422 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:26.115434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:26.117371 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":112,"threads_started":1}
W20260811 18:50:26.117705 6346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:26.111794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:48617 (local address 127.27.115.254:37875)
0811 18:50:26.112073 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:26.112077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:26.112192 (+ 115us) server_negotiation.cc:407] Connection header received
0811 18:50:26.112356 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:26.112359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:26.112441 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:26.112550 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:26.113050 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.113604 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.114468 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.114703 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.114798 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.117550 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:48617: BlockingRecv error: recv got EOF from 127.27.115.193:48617 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":73,"threads_started":1}
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:26.325073 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:26.319226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:43701)
0811 18:50:26.319531 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:26.319544 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:26.319649 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:26.319876 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:26.319878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:26.320073 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:26.320197 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.320203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.320973 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.320976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.321599 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.321610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.322236 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.322240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.322310 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.322862 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:26.322875 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:26.324937 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":111,"threads_started":1}
W20260811 18:50:26.325320 6346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:26.319321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:43701 (local address 127.27.115.254:37875)
0811 18:50:26.319476 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:26.319480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:26.319613 (+ 133us) server_negotiation.cc:407] Connection header received
0811 18:50:26.319717 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:26.319719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:26.319765 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:26.319837 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:26.320312 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.320844 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.321750 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.321994 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.322090 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.325177 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:43701: BlockingRecv error: recv got EOF from 127.27.115.194:43701 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:26 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:26.541739 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:26.536058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:57801)
0811 18:50:26.536356 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:26.536369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:26.536478 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:26.536719 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:26.536721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:26.536911 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:26.537014 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.537019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.537864 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.537866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.538517 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:26.538528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.539048 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.539052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:26.539120 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.539658 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:26.539669 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:26.541579 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
W20260811 18:50:26.541960 6346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:26.536090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:57801 (local address 127.27.115.254:37875)
0811 18:50:26.536234 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:26.536238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:26.536442 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:50:26.536557 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:26.536560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:26.536607 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:26.536697 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:26.537125 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.537745 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.538640 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:26.538923 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:26.539010 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:26.541817 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:57801: BlockingRecv error: recv got EOF from 127.27.115.195:57801 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:27.124894 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:27.118645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:60549)
0811 18:50:27.118965 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:27.118982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:27.119091 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:27.119502 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:27.119508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:27.119770 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:27.119923 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.119931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.120752 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.120757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.121340 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.121349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.121857 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.121861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.121955 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.122546 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:27.122560 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:27.124709 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":107,"threads_started":1}
W20260811 18:50:27.125027 6350 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:27.118798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60549 (local address 127.27.115.254:37875)
0811 18:50:27.119111 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:27.119116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:27.119132 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:27.119260 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:27.119262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:27.119356 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:27.119454 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:27.120058 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.120628 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.121474 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.121731 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.121850 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.124880 (+ 3030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:60549: BlockingRecv error: recv got EOF from 127.27.115.193:60549 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:27.333482 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:27.325979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:35455)
0811 18:50:27.326400 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:27.326415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:27.326561 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:27.326917 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:27.326920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:27.327204 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:27.327337 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.327343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.328509 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.328512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.329153 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.329165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.330102 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.330106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.330267 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.330998 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:27.331022 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:27.333286 (+ 2264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":131,"threads_started":1}
W20260811 18:50:27.333583 6350 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:27.326064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:35455 (local address 127.27.115.254:37875)
0811 18:50:27.326271 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:27.326277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:27.326501 (+ 224us) server_negotiation.cc:407] Connection header received
0811 18:50:27.326685 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:27.326690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:27.326782 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:27.326873 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:27.327454 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.328389 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.329298 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.329646 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.329755 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.333459 (+ 3704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:35455: BlockingRecv error: recv got EOF from 127.27.115.194:35455 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:27 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:27.548720 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:27.542865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:51619)
0811 18:50:27.543208 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:27.543223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:27.543366 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:27.543644 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:27.543647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:27.543874 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:27.543972 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.543978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.544741 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.544746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.545310 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:27.545318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.545782 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.545786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:27.545871 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.546502 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:27.546514 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:27.548533 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":95,"threads_started":1}
W20260811 18:50:27.548790 6350 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:27.542967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:51619 (local address 127.27.115.254:37875)
0811 18:50:27.543148 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:27.543153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:27.543306 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:50:27.543475 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:27.543479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:27.543532 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:27.543636 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:27.544093 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.544629 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.545434 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:27.545665 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:27.545775 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:27.548678 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:51619: BlockingRecv error: recv got EOF from 127.27.115.195:51619 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:28.132270 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:28.126216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:60143)
0811 18:50:28.126541 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:28.126556 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:28.126660 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:28.126939 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:28.126941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:28.127148 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:28.127247 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.127252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.128152 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.128155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.128776 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.128785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.129293 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.129298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.129377 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.130007 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:28.130020 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:28.132126 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
W20260811 18:50:28.132419 6354 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:28.126311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:60143 (local address 127.27.115.254:37875)
0811 18:50:28.126618 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:28.126622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:28.126638 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:28.126738 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:28.126747 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:28.126791 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:28.126874 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:28.127356 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.128046 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.128884 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.129136 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.129239 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.132287 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:60143: BlockingRecv error: recv got EOF from 127.27.115.193:60143 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":102,"threads_started":1}
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:28.341205 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:28.334716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:58559)
0811 18:50:28.335297 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:28.335330 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:28.335452 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:28.335807 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:28.335812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:28.336064 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:28.336187 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.336193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.336998 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.337000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.337578 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.337602 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.338117 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.338120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.338247 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.338806 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:28.338828 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:28.341035 (+ 2207us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":100,"threads_started":1}
W20260811 18:50:28.341326 6354 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:28.334969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:58559 (local address 127.27.115.254:37875)
0811 18:50:28.335153 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:28.335157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:28.335450 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:50:28.335540 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:28.335545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:28.335605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:28.335725 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:28.336323 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.336895 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.337713 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.337994 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.338174 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.341203 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:58559: BlockingRecv error: recv got EOF from 127.27.115.194:58559 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:28 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:28.556017 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:28.549963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:33697)
0811 18:50:28.550292 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:28.550307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:28.550442 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:28.550717 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:28.550721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:28.550952 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:28.551044 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.551050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.551811 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.551815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.552494 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:28.552503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.552961 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.552964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:28.553047 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.553528 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:28.553540 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:28.555837 (+ 2297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
W20260811 18:50:28.556077 6354 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:28.550048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:33697 (local address 127.27.115.254:37875)
0811 18:50:28.550227 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:28.550233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:28.550392 (+ 159us) server_negotiation.cc:407] Connection header received
0811 18:50:28.550548 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:28.550551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:28.550606 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:28.550694 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:28.551164 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.551694 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.552621 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:28.552845 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:28.552944 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:28.555965 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:33697: BlockingRecv error: recv got EOF from 127.27.115.195:33697 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:29.139344 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:29.133254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:35513)
0811 18:50:29.133551 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:29.133564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:29.133660 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:29.133996 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:29.133999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:29.134253 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:29.134361 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.134367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.135204 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.135207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.135768 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.135777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.136238 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.136241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.136302 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.136828 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:29.136840 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:29.139150 (+ 2310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":114,"threads_started":1}
W20260811 18:50:29.139460 6360 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:29.133474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:35513 (local address 127.27.115.254:37875)
0811 18:50:29.133714 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:29.133717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:29.133731 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:50:29.133771 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:29.133772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:29.133828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:29.133915 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:29.134524 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.135075 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.135886 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.136125 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.136213 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.139319 (+ 3106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:35513: BlockingRecv error: recv got EOF from 127.27.115.193:35513 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":85,"threads_started":1}
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:29.348438 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:29.342224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:60663)
0811 18:50:29.342627 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:29.342647 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:29.342827 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:29.343140 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:29.343143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:29.343369 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:29.343463 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.343468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.344272 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.344277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.345013 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.345023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.345522 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.345525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.345609 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.346243 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:29.346256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:29.348262 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":100,"threads_started":1}
W20260811 18:50:29.348546 6360 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:29.342397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:60663 (local address 127.27.115.254:37875)
0811 18:50:29.342630 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:29.342636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:29.342756 (+ 120us) server_negotiation.cc:407] Connection header received
0811 18:50:29.342954 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:29.342957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:29.343002 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:29.343109 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:29.343590 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.344149 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.345156 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.345380 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.345516 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.348429 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:60663: BlockingRecv error: recv got EOF from 127.27.115.194:60663 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:29 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:29.564399 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:29.557191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:44309)
0811 18:50:29.557545 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:29.557560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:29.557701 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:29.557972 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:29.557974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:29.558233 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:29.558345 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.558351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.559287 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.559290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.559959 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:29.559970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.560577 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.560581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:29.560672 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.561467 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:29.561482 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:29.564152 (+ 2670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":143,"threads_started":1}
W20260811 18:50:29.564839 6360 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:29.557250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:44309 (local address 127.27.115.254:37875)
0811 18:50:29.557427 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:29.557432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:29.557668 (+ 236us) server_negotiation.cc:407] Connection header received
0811 18:50:29.557795 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:29.557798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:29.557853 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:29.557933 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:29.558478 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.559108 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.560137 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.560406 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.560524 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.564365 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:44309: BlockingRecv error: recv got EOF from 127.27.115.195:44309 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:30.151222 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.140553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:44443)
0811 18:50:30.140983 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.140998 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.141112 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.141428 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.141431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.141656 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.141768 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.141774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.142933 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.142936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.143789 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.143804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.144394 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.144397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.144468 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.148464 (+ 3996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.148482 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.151037 (+ 2555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":104,"threads_started":1}
W20260811 18:50:30.151378 6364 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:30.140693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:44443 (local address 127.27.115.254:37875)
0811 18:50:30.140983 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.140987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.141080 (+ 93us) server_negotiation.cc:407] Connection header received
0811 18:50:30.141196 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.141198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.141246 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.141332 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.141937 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.142742 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.143928 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.144273 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.144361 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.151240 (+ 6879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:44443: BlockingRecv error: recv got EOF from 127.27.115.193:44443 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":67,"threads_started":1}
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:30.355799 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.349495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:46333)
0811 18:50:30.349858 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.349871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.350032 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.350438 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.350440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.350634 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.350752 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.350758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.351707 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.351713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.352548 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.352560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.353082 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.353085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.353162 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.353695 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.353709 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.355672 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":107,"threads_started":1}
W20260811 18:50:30.356092 6364 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:30.349574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:46333 (local address 127.27.115.254:37875)
0811 18:50:30.349805 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.349809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.349984 (+ 175us) server_negotiation.cc:407] Connection header received
0811 18:50:30.350251 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.350254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.350306 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.350393 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.350895 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.351582 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.352698 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.352958 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.353042 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.355958 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:46333: BlockingRecv error: recv got EOF from 127.27.115.194:46333 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:30 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:30.571461 6366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.565507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:35939)
0811 18:50:30.565844 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.565858 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.565966 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.566238 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.566241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.566446 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.566543 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.566549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.567320 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.567323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.567898 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.567908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.568388 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.568390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.568449 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.569036 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.569048 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.571329 (+ 2281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":103,"threads_started":1}
W20260811 18:50:30.571694 6364 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:30.565580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:35939 (local address 127.27.115.254:37875)
0811 18:50:30.565771 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.565776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.565938 (+ 162us) server_negotiation.cc:407] Connection header received
0811 18:50:30.566036 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.566038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.566085 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.566222 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.566654 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.567190 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.568034 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.568276 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.568362 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.571552 (+ 3190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:35939: BlockingRecv error: recv got EOF from 127.27.115.195:35939 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:31.159091 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.152373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:37427)
0811 18:50:31.152667 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.152690 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.152990 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.153546 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.153558 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.153753 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.153863 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.153869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.154801 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.154805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.155504 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.155519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.156092 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.156096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.156189 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.156807 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.156820 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.158892 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":115,"threads_started":1}
W20260811 18:50:31.159206 6368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:31.152620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:37427 (local address 127.27.115.254:37875)
0811 18:50:31.152868 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.152872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.152911 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:50:31.153220 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.153223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.153277 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.153376 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.154000 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.154633 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.155686 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.155955 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.156076 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.159067 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:37427: BlockingRecv error: recv got EOF from 127.27.115.193:37427 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:31.363742 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.356707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:33191)
0811 18:50:31.357047 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.357063 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.357182 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.357523 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.357526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.357743 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.357859 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.357865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.358951 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.358954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.359646 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.359660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.360221 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.360224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.360305 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.361023 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.361039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.363581 (+ 2542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
W20260811 18:50:31.364039 6368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:31.356884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:33191 (local address 127.27.115.254:37875)
0811 18:50:31.357083 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.357087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.357123 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:50:31.357294 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.357296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.357348 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.357434 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.357982 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.358805 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.359797 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.360051 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.360148 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.363819 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:33191: BlockingRecv error: recv got EOF from 127.27.115.194:33191 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:31 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:31.578619 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.572507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:59333)
0811 18:50:31.572825 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.572839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.572946 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.573219 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.573221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.573408 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.573509 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.573515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.574406 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.574409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.574979 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.574988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.575543 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.575547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.575655 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.576267 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.576281 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.578428 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":118,"threads_started":1}
W20260811 18:50:31.578713 6368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:31.572686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:59333 (local address 127.27.115.254:37875)
0811 18:50:31.572849 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.572853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.572901 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:50:31.573050 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.573060 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.573110 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.573187 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.573689 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.574270 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.575175 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.575405 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.575524 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.578594 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:59333: BlockingRecv error: recv got EOF from 127.27.115.195:59333 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:32.167774 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.160168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:50807)
0811 18:50:32.160552 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.160575 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.160719 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.161200 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.161203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.161429 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:32.161569 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.161576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.162793 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.162797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.163534 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.163553 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.164150 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.164154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.164301 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.165074 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:32.165104 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:32.167563 (+ 2459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":101,"threads_started":1}
W20260811 18:50:32.168074 6372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:32.160368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:50807 (local address 127.27.115.254:37875)
0811 18:50:32.160803 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.160808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.160831 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:50:32.160902 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.160905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.160969 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.161104 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:32.161815 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.162627 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.163714 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.164004 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.164125 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.167772 (+ 3647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:50807: BlockingRecv error: recv got EOF from 127.27.115.193:50807 (error 108)
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":107,"threads_started":1}
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:32.370884 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.364937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:46883)
0811 18:50:32.365256 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.365272 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.365520 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.365818 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.365820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.366023 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:32.366185 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.366198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.366962 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.366964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.367547 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.367554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.368032 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.368034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.368098 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.368626 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:32.368638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:32.370729 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":110,"threads_started":1}
W20260811 18:50:32.371099 6372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:32.365013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:46883 (local address 127.27.115.254:37875)
0811 18:50:32.365184 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.365190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.365388 (+ 198us) server_negotiation.cc:407] Connection header received
0811 18:50:32.365632 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.365634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.365685 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.365764 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:32.366311 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.366860 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.367677 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.367917 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.368010 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.370928 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:46883: BlockingRecv error: recv got EOF from 127.27.115.194:46883 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:32 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:32.585243 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.579520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:52653)
0811 18:50:32.579854 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.579878 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.580019 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.580336 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.580338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.580554 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:32.580653 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.580659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.581434 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.581439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.582007 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.582016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.582525 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.582528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.582614 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.583168 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:32.583180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:32.585089 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":93,"threads_started":1}
W20260811 18:50:32.585374 6372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:32.579657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:52653 (local address 127.27.115.254:37875)
0811 18:50:32.579829 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.579834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.579958 (+ 124us) server_negotiation.cc:407] Connection header received
0811 18:50:32.580122 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.580125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.580205 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.580295 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:32.580799 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.581322 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.582169 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.582396 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.582507 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.585247 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:52653: BlockingRecv error: recv got EOF from 127.27.115.195:52653 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:33.179306 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:33.169186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:42213)
0811 18:50:33.169856 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:33.169878 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:33.170071 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:33.170727 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:33.170730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:33.171024 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:33.171189 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.171198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.172207 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.172210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.172879 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.172892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.173615 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.173618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.173725 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.174830 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:33.174850 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:33.178907 (+ 4057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":251,"threads_started":1}
W20260811 18:50:33.180212 6377 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:33.169479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:42213 (local address 127.27.115.254:37875)
0811 18:50:33.169796 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:33.169800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:33.170114 (+ 314us) server_negotiation.cc:407] Connection header received
0811 18:50:33.170357 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:33.170362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:33.170438 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:33.170556 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:33.171386 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.172030 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.173115 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.173421 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.173564 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.179779 (+ 6215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:42213: BlockingRecv error: recv got EOF from 127.27.115.193:42213 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":77,"threads_started":1}
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:33.377978 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:33.371950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:33347)
0811 18:50:33.372319 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:33.372333 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:33.372451 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:33.372723 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:33.372725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:33.372951 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:33.373064 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.373069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.373901 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.373904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.374582 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.374595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.375108 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.375111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.375194 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.375884 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:33.375898 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:33.377844 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":82,"threads_started":1}
W20260811 18:50:33.378284 6377 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:33.372116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:33347 (local address 127.27.115.254:37875)
0811 18:50:33.372320 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:33.372324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:33.372421 (+ 97us) server_negotiation.cc:407] Connection header received
0811 18:50:33.372544 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:33.372546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:33.372594 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:33.372674 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:33.373212 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.373762 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.374744 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.374982 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.375075 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.378123 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:33347: BlockingRecv error: recv got EOF from 127.27.115.194:33347 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:33 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:33.593245 6379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:33.586533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:34659)
0811 18:50:33.586828 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:33.586847 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:33.586938 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:33.587202 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:33.587204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:33.587421 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:33.587521 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.587527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.588361 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.588364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.589335 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:33.589348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.589906 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.589912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:33.590013 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.590659 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:33.590672 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:33.593079 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
W20260811 18:50:33.593469 6377 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:33.586767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:34659 (local address 127.27.115.254:37875)
0811 18:50:33.586959 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:33.586963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:33.586980 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:33.587038 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:33.587041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:33.587085 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:33.587185 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:33.587670 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.588220 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.589491 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:33.589786 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:33.589879 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:33.593292 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:34659: BlockingRecv error: recv got EOF from 127.27.115.195:34659 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:34.187415 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:34.180592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:58083)
0811 18:50:34.180920 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:34.180934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:34.181036 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:34.181405 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:34.181408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:34.181627 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:34.181750 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.181756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.182752 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.182756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.183362 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.183371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.183867 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.183870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.183942 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.184507 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:34.184518 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:34.187245 (+ 2727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":107,"threads_started":1}
W20260811 18:50:34.187604 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:34.180700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:58083 (local address 127.27.115.254:37875)
0811 18:50:34.181091 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:34.181095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:34.181112 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:34.181157 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:34.181159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:34.181215 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:34.181316 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:34.181924 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.182589 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.183490 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.183729 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.183819 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.187462 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:58083: BlockingRecv error: recv got EOF from 127.27.115.193:58083 (error 108)
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":149,"threads_started":1}
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:34.385473 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:34.379011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:43201)
0811 18:50:34.379411 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:34.379431 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:34.379581 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:34.379878 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:34.379881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:34.380138 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:34.380241 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.380248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.381096 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.381100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.381706 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.381717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.382349 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.382353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.382441 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.383029 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:34.383042 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:34.385311 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":141,"threads_started":1}
W20260811 18:50:34.385632 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:34.379133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:43201 (local address 127.27.115.254:37875)
0811 18:50:34.379326 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:34.379331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:34.379519 (+ 188us) server_negotiation.cc:407] Connection header received
0811 18:50:34.379700 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:34.379707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:34.379755 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:34.379852 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:34.380384 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.380962 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.381854 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.382097 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.382252 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.385496 (+ 3244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:43201: BlockingRecv error: recv got EOF from 127.27.115.194:43201 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:34 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:34.601881 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:34.594381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:55437)
0811 18:50:34.594816 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:34.594835 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:34.594946 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:34.595366 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:34.595369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:34.595807 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:34.595915 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.595922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.596754 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.596761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.597326 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:34.597335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.597825 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.597828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:34.597917 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.598945 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:34.598959 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:34.601750 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
W20260811 18:50:34.602337 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:34.594548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:55437 (local address 127.27.115.254:37875)
0811 18:50:34.594975 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:34.594979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:34.594999 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:50:34.595046 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:34.595049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:34.595109 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:34.595212 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:34.596071 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.596632 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.597460 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:34.597695 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:34.597815 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:34.602172 (+ 4357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:55437: BlockingRecv error: recv got EOF from 127.27.115.195:55437 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:35.199726 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.188756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:34673)
0811 18:50:35.189222 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.189235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.189379 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.189664 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.189668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.189897 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.189996 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.190002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.190829 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.190833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.191432 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.191441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.191970 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.191973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.192059 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.197123 (+ 5064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.197142 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.199539 (+ 2397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
W20260811 18:50:35.200039 6385 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:35.188847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:34673 (local address 127.27.115.254:37875)
0811 18:50:35.189157 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.189163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.189324 (+ 161us) server_negotiation.cc:407] Connection header received
0811 18:50:35.189482 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.189487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.189537 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.189641 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.190116 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.190706 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.191562 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.191819 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.191961 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.199759 (+ 7798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:34673: BlockingRecv error: recv got EOF from 127.27.115.193:34673 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":125,"threads_started":1}
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:35.392565 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.386787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:40117)
0811 18:50:35.387133 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.387150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.387327 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.387617 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.387620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.387847 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.387940 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.387946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.388717 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.388722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.389280 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.389289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.389765 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.389768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.389848 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.390511 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.390523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.392437 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
W20260811 18:50:35.392879 6385 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:35.386871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:40117 (local address 127.27.115.254:37875)
0811 18:50:35.387109 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.387114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.387241 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:50:35.387435 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.387439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.387494 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.387584 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.388069 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.388599 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.389407 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.389639 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.389763 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.392625 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:40117: BlockingRecv error: recv got EOF from 127.27.115.194:40117 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:35 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:35.608556 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.602804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:48037)
0811 18:50:35.603120 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.603134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.603236 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.603595 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.603597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.603787 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.603887 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.603892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.604658 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.604660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.605255 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.605265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.605755 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.605758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.605820 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.606437 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.606455 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.608426 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
W20260811 18:50:35.608850 6385 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:35.602911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:48037 (local address 127.27.115.254:37875)
0811 18:50:35.603111 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.603115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.603207 (+ 92us) server_negotiation.cc:407] Connection header received
0811 18:50:35.603359 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.603361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.603430 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.603509 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.603998 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.604547 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.605395 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.605636 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.605721 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.608602 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:48037: BlockingRecv error: recv got EOF from 127.27.115.195:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:36.207393 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.200828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:50677)
0811 18:50:36.201204 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:36.201221 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:36.201344 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:36.201653 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:36.201655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:36.201861 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:36.201996 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.202004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.202957 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.202960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.203582 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.203592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.204123 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.204127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.204224 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.204892 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:36.204908 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:36.207246 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
W20260811 18:50:36.207583 6389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:36.201033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:50677 (local address 127.27.115.254:37875)
0811 18:50:36.201349 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.201353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.201369 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:36.201417 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.201419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.201464 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.201554 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.202215 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.202805 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.203738 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.203989 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.204112 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.207405 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:50677: BlockingRecv error: recv got EOF from 127.27.115.193:50677 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":122,"threads_started":1}
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:36.399530 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.393481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:45385)
0811 18:50:36.393789 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:36.393803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:36.393907 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:36.394168 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:36.394170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:36.394399 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:36.394501 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.394507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.395255 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.395257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.395818 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.395825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.396330 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.396334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.396422 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.397145 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:36.397165 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:36.399374 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":114,"threads_started":1}
W20260811 18:50:36.399737 6389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:36.393620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:45385 (local address 127.27.115.254:37875)
0811 18:50:36.393782 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.393792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.393876 (+ 84us) server_negotiation.cc:407] Connection header received
0811 18:50:36.393989 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.393990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.394035 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.394106 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.394607 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.395155 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.395959 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.396186 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.396312 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.399563 (+ 3251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:45385: BlockingRecv error: recv got EOF from 127.27.115.194:45385 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:36 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:36.616272 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.609749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:43359)
0811 18:50:36.610084 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:36.610100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:36.610247 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:36.610572 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:36.610575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:36.610788 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:36.610890 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.610895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.611831 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.611835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.612681 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.612694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.613376 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.613379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.613441 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.613979 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:36.613991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:36.616117 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":130,"threads_started":1}
W20260811 18:50:36.616408 6389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:36.609912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:43359 (local address 127.27.115.254:37875)
0811 18:50:36.610067 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.610072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.610200 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:50:36.610352 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.610354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.610406 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.610495 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.611021 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.611664 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.612871 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.613231 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.613345 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.616281 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:43359: BlockingRecv error: recv got EOF from 127.27.115.195:43359 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:37.214421 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:37.208462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:53075)
0811 18:50:37.208800 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:37.208816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:37.208919 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:37.209204 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:37.209206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:37.209417 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:37.209521 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.209526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.210353 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.210356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.210923 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.210932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.211457 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.211460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.211524 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.212129 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:37.212141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:37.214236 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
W20260811 18:50:37.214689 6393 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:37.208544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:53075 (local address 127.27.115.254:37875)
0811 18:50:37.208884 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:37.208888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:37.208905 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:50:37.209022 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:37.209025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:37.209073 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:37.209156 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:37.209650 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.210226 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.211076 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.211328 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.211425 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.214441 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:53075: BlockingRecv error: recv got EOF from 127.27.115.193:53075 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":69,"threads_started":1}
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:37.406704 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:37.400694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:50023)
0811 18:50:37.400993 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:37.401007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:37.401117 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:37.401391 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:37.401393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:37.401603 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:37.401709 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.401715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.402618 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.402621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.403263 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.403273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.403729 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.403732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.403794 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.404335 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:37.404348 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:37.406502 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":119,"threads_started":1}
W20260811 18:50:37.406810 6393 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:37.400777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:50023 (local address 127.27.115.254:37875)
0811 18:50:37.400940 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:37.400944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:37.401114 (+ 170us) server_negotiation.cc:407] Connection header received
0811 18:50:37.401214 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:37.401217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:37.401268 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:37.401363 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:37.401825 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.402482 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.403387 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.403620 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.403702 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.406678 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:50023: BlockingRecv error: recv got EOF from 127.27.115.194:50023 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:37 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.195@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:37.623847 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:37.617206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.195:51539)
0811 18:50:37.617504 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:37.617517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:37.617622 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:37.617956 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:37.617959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:37.618238 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:37.618359 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.618365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.619259 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.619263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.619860 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:37.619870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.620439 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.620442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:37.620522 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.621425 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:37.621441 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:37.623689 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
W20260811 18:50:37.624060 6393 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:37.617405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:51539 (local address 127.27.115.254:37875)
0811 18:50:37.617553 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:37.617557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:37.617572 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:50:37.617699 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:37.617701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:37.617749 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:37.617853 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:37.618523 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.619092 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.620034 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:37.620272 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:37.620366 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:37.623881 (+ 3515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.195:51539: BlockingRecv error: recv got EOF from 127.27.115.195:51539 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 18:50:38 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:38 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.193@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:38.222368 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:38.215584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.193:38725)
0811 18:50:38.215981 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:38.215997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:38.216186 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:38.216583 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:38.216586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:38.216818 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:38.216928 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:38.216935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.218042 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.218046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:38.218712 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:38.218724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.219239 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.219243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:38.219317 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.219948 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:38.219965 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:38.222177 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":123,"threads_started":1}
W20260811 18:50:38.222694 6397 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:38.215799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:38725 (local address 127.27.115.254:37875)
0811 18:50:38.216078 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:38.216082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:38.216103 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:50:38.216351 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:38.216354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:38.216408 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:38.216522 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:38.217119 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.217878 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.218857 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.219097 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.219190 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.222524 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.193:38725: BlockingRecv error: recv got EOF from 127.27.115.193:38725 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":70,"threads_started":1}
Aug 11 18:50:38 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:50:38 dist-test-slave-9jjr krb5kdc[5168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.115.194@KRBTEST.COM for oryx/127.27.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:50:38.413781 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:38.407931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37875 (local address 127.27.115.194:37015)
0811 18:50:38.408227 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:38.408241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:38.408380 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:38.408664 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:38.408667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:38.408892 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:38.408983 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:38.408989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.409822 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.409827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:38.410468 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:38.410477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.410951 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.410954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:38.411039 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.411642 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:38.411654 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:38.413587 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.115.254:37875: Server oryx/127.27.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":85,"threads_started":1}
W20260811 18:50:38.413870 6397 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:38.408003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:37015 (local address 127.27.115.254:37875)
0811 18:50:38.408172 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:38.408177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:38.408325 (+ 148us) server_negotiation.cc:407] Connection header received
0811 18:50:38.408488 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:38.408491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:38.408546 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:38.408638 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:38.409156 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.409701 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.410604 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.410829 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.410943 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.413747 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.115.194:37015: BlockingRecv error: recv got EOF from 127.27.115.194:37015 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:50:38.504138 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5840
I20260811 18:50:38.514024 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6021
I20260811 18:50:38.524868 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6180
I20260811 18:50:38.533628 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 5756
2026-08-11T18:50:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18444 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T18:50:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:38Z Disabled control of system clock
I20260811 18:50:38.570984 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35667
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:50:38.696560 6403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:38.696888 6403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:38.696925 6403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:38.701493 6403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:38.701583 6403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:38.701599 6403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:38.701618 6403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:38.701634 6403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:38.706895 6403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:35667
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:38.708125 6403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:38.709585 6403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:38.719807 6413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:38.719945 6403 server_base.cc:1034] running on GCE node
W20260811 18:50:38.719800 6409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:38.719866 6410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:38.720942 6403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:38.722433 6403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:38.723630 6403 hybrid_clock.cc:648] HybridClock initialized: now 1786474238723603 us; error 49 us; skew 500 ppm
I20260811 18:50:38.726106 6403 webserver.cc:460] Webserver started at http://127.27.115.254:37055/ using document root <none> and password file <none>
I20260811 18:50:38.726984 6403 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:38.727069 6403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:38.727267 6403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:38.729544 6403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "5e35bc433be84f0b86b77af0763508b0"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-9jjr"
server_key: "e597b733f3aea5e47d1c47b2b011a99a"
server_key_iv: "57d4aa6a79ee67fdad71c02298a4af2b"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.730201 6403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "5e35bc433be84f0b86b77af0763508b0"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-9jjr"
server_key: "e597b733f3aea5e47d1c47b2b011a99a"
server_key_iv: "57d4aa6a79ee67fdad71c02298a4af2b"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.734390 6403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 18:50:38.737058 6419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:38.738560 6403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:38.738689 6403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "5e35bc433be84f0b86b77af0763508b0"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-9jjr"
server_key: "e597b733f3aea5e47d1c47b2b011a99a"
server_key_iv: "57d4aa6a79ee67fdad71c02298a4af2b"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.738811 6403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:38.749402 6403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:38.750288 6403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:38.750471 6403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:38.758675 6403 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:35667
I20260811 18:50:38.758698 6482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:35667 every 8 connection(s)
I20260811 18:50:38.759742 6403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:38.763345 6484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:38.768610 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6403
I20260811 18:50:38.768759 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:38.769059 28111 external_mini_cluster.cc:1428] Setting key cfbd9d19d9848fce57366d989a3b83b0
I20260811 18:50:38.769794 6484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0: Bootstrap starting.
I20260811 18:50:38.772377 6484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:38.773342 6484 log.cc:824] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:38.775969 6484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0: No bootstrap required, opened a new log
I20260811 18:50:38.780155 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:38.770655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.27.115.254:35667)
0811 18:50:38.771374 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:38.771389 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:38.771427 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:50:38.772091 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:38.772107 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:38.772559 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:38.772872 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:38.773250 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.774369 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.775635 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.776315 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.776466 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.777630 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:38.777637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:38.777654 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:38.778395 (+ 741us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:38.778475 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:38.778502 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:38.778615 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:38.778842 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:50:38.779250 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":166,"threads_started":1}
I20260811 18:50:38.780653 6484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } }
I20260811 18:50:38.780972 6484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:38.781023 6484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e35bc433be84f0b86b77af0763508b0, State: Initialized, Role: FOLLOWER
I20260811 18:50:38.781517 6484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } }
I20260811 18:50:38.781668 6484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:38.781740 6484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:38.781852 6484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:38.783129 6484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } }
I20260811 18:50:38.783493 6484 leader_election.cc:302] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e35bc433be84f0b86b77af0763508b0; no voters:
I20260811 18:50:38.783841 6484 leader_election.cc:288] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:38.784128 6490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:38.784451 6490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5e35bc433be84f0b86b77af0763508b0, State: Running, Role: LEADER
I20260811 18:50:38.784788 6490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } }
I20260811 18:50:38.785223 6484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:38.786630 6491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e35bc433be84f0b86b77af0763508b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } } }
I20260811 18:50:38.786798 6491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:38.787361 6499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:38.787946 6492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e35bc433be84f0b86b77af0763508b0. Latest consensus state: current_term: 1 leader_uuid: "5e35bc433be84f0b86b77af0763508b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e35bc433be84f0b86b77af0763508b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35667 } } }
I20260811 18:50:38.788182 6492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:38.791364 6499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:38.798736 6499 catalog_manager.cc:1347] Generated new cluster ID: c61d366586a44464a4567b15030ea08a
I20260811 18:50:38.798871 6499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:38.813578 6499 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:38.814985 6499 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:38.826277 6499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e35bc433be84f0b86b77af0763508b0: Generated new TSK 0
I20260811 18:50:38.827169 6499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:38.835443 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--builtin_ntp_servers=127.27.115.212:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:50:38.985069 6509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:38.985549 6509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:38.985597 6509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:38.990209 6509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:38.990288 6509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:38.990345 6509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:38.995498 6509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:38.996958 6509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:38.998728 6509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:39.009909 6517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.009958 6515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.009964 6514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:39.010447 6509 server_base.cc:1034] running on GCE node
I20260811 18:50:39.010928 6509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:39.011647 6509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:39.012820 6509 hybrid_clock.cc:648] HybridClock initialized: now 1786474239012785 us; error 47 us; skew 500 ppm
I20260811 18:50:39.014858 6509 webserver.cc:460] Webserver started at http://127.27.115.193:40131/ using document root <none> and password file <none>
I20260811 18:50:39.015473 6509 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:39.015570 6509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:39.015836 6509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:39.018038 6509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "050f5ac8900143ee9ff0df2c938e1fa7"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "a41e1bd32a1bc967945ca158a07a8833"
server_key_iv: "0a281882479aec09b9b6b82f64fad420"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.018798 6509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "050f5ac8900143ee9ff0df2c938e1fa7"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "a41e1bd32a1bc967945ca158a07a8833"
server_key_iv: "0a281882479aec09b9b6b82f64fad420"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.022725 6509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:50:39.025665 6523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.027344 6509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:39.027549 6509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "050f5ac8900143ee9ff0df2c938e1fa7"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "a41e1bd32a1bc967945ca158a07a8833"
server_key_iv: "0a281882479aec09b9b6b82f64fad420"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.027722 6509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:39.047346 6509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:39.048141 6509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:39.048369 6509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:39.049009 6509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:39.050047 6509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:39.050120 6509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.050209 6509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:39.050254 6509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.060002 6509 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:38597
I20260811 18:50:39.060060 6650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:38597 every 8 connection(s)
I20260811 18:50:39.061084 6509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:39.066175 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6509
I20260811 18:50:39.066339 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:39.066712 28111 external_mini_cluster.cc:1428] Setting key 8e3431f90031e34dbe768b728a50a219
I20260811 18:50:39.069123 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--builtin_ntp_servers=127.27.115.212:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:50:39.073307 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.063385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:47683 (local address 127.27.115.254:35667)
0811 18:50:39.063581 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:39.063586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:39.064703 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 18:50:39.066656 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:39.066659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:39.066712 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:39.066824 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:39.068014 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.068732 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.070802 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.071055 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.071215 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.072595 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:39.072599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:39.072601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:39.072619 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:39.072641 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:39.072650 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:39.072760 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:39.073014 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 18:50:39.073094 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:50:39.074848 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.063797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35667 (local address 127.27.115.193:47683)
0811 18:50:39.064532 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:39.064578 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:39.066403 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:39.067046 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:39.067055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:39.067178 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:39.067850 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.067866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.068914 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.068920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.070611 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.070634 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.071280 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.071285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.071491 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.072225 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:39.072227 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:39.072279 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:39.072369 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:39.072798 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:39.072814 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:39.072824 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:50:39.073003 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 18:50:39.073426 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":126,"threads_started":1}
I20260811 18:50:39.076853 6651 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35667
I20260811 18:50:39.077206 6651 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:39.078045 6651 heartbeater.cc:511] Master 127.27.115.254:35667 requested a full tablet report, sending...
I20260811 18:50:39.080432 6439 ts_manager.cc:194] Registered new tserver with Master: 050f5ac8900143ee9ff0df2c938e1fa7 (127.27.115.193:38597)
I20260811 18:50:39.082367 6439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.193:47683
W20260811 18:50:39.199527 6655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:39.199841 6655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:39.199882 6655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:39.204063 6655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:39.204146 6655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:39.204205 6655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:39.209172 6655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:39.210364 6655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:39.211786 6655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:39.222369 6664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.222370 6662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.222354 6661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:39.222463 6655 server_base.cc:1034] running on GCE node
I20260811 18:50:39.223104 6655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:39.223886 6655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:39.225101 6655 hybrid_clock.cc:648] HybridClock initialized: now 1786474239225085 us; error 32 us; skew 500 ppm
I20260811 18:50:39.227305 6655 webserver.cc:460] Webserver started at http://127.27.115.194:35897/ using document root <none> and password file <none>
I20260811 18:50:39.227950 6655 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:39.228051 6655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:39.228310 6655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:39.230631 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "464c152b945a44f8aaa5978d0637a2b9"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "61e88d5877bc253ce8b627f5aa4faf59"
server_key_iv: "b7a4de819f2c0faa194c059bbb02365c"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.231259 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "464c152b945a44f8aaa5978d0637a2b9"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "61e88d5877bc253ce8b627f5aa4faf59"
server_key_iv: "b7a4de819f2c0faa194c059bbb02365c"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.235352 6655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 18:50:39.238176 6670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.239535 6655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:50:39.239694 6655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "464c152b945a44f8aaa5978d0637a2b9"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "61e88d5877bc253ce8b627f5aa4faf59"
server_key_iv: "b7a4de819f2c0faa194c059bbb02365c"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.239835 6655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:39.259264 6655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:39.260053 6655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:39.260274 6655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:39.260943 6655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:39.261961 6655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:39.262036 6655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.262108 6655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:39.262188 6655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.271909 6655 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:44331
I20260811 18:50:39.271931 6799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:44331 every 8 connection(s)
I20260811 18:50:39.272907 6655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:39.278863 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6655
I20260811 18:50:39.279000 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:39.279397 28111 external_mini_cluster.cc:1428] Setting key 4bc2a7725d960f16c29c0ddf80658573
I20260811 18:50:39.281464 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--builtin_ntp_servers=127.27.115.212:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:50:39.283239 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.274860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:51835 (local address 127.27.115.254:35667)
0811 18:50:39.275005 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:39.275009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:39.276194 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 18:50:39.277703 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:39.277706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:39.277772 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:39.277881 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:39.278909 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.279582 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.281023 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.281258 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.281375 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.282693 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:39.282697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:39.282700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:39.282714 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:39.282742 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:39.282750 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:39.282832 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:39.283008 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 18:50:39.283092 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:50:39.284215 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.275282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35667 (local address 127.27.115.194:51835)
0811 18:50:39.276019 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:39.276063 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:39.277459 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:39.278079 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:39.278087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:39.278240 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:39.278760 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.278774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.279735 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.279738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.280891 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.280903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.281423 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.281427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.281635 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.282397 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:39.282399 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:39.282444 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:39.282516 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:39.282834 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:39.282849 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:39.282855 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:50:39.282997 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 18:50:39.283333 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":112,"threads_started":1}
I20260811 18:50:39.285506 6801 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35667
I20260811 18:50:39.285781 6801 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:39.286589 6801 heartbeater.cc:511] Master 127.27.115.254:35667 requested a full tablet report, sending...
I20260811 18:50:39.287932 6439 ts_manager.cc:194] Registered new tserver with Master: 464c152b945a44f8aaa5978d0637a2b9 (127.27.115.194:44331)
I20260811 18:50:39.288540 6439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.194:51835
W20260811 18:50:39.403311 6804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:39.403614 6804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:39.403656 6804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:39.408061 6804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:39.408133 6804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:39.408191 6804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:39.413308 6804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:39.414665 6804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:39.416253 6804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:39.427078 6814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.427086 6811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.427161 6812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:39.427497 6804 server_base.cc:1034] running on GCE node
I20260811 18:50:39.427983 6804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:39.428696 6804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:39.429874 6804 hybrid_clock.cc:648] HybridClock initialized: now 1786474239429861 us; error 26 us; skew 500 ppm
I20260811 18:50:39.432199 6804 webserver.cc:460] Webserver started at http://127.27.115.195:42767/ using document root <none> and password file <none>
I20260811 18:50:39.432860 6804 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:39.432929 6804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:39.433198 6804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:39.435434 6804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "cd2aea88c0dd40d6a319ef2d7904a9db"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "054b08930200cfa1b353176683a765be"
server_key_iv: "f67c0cc4e5dcb63036f4a2a7f016adcc"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.436028 6804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "cd2aea88c0dd40d6a319ef2d7904a9db"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "054b08930200cfa1b353176683a765be"
server_key_iv: "f67c0cc4e5dcb63036f4a2a7f016adcc"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.439976 6804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:39.443018 6820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.444739 6804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:39.444926 6804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "cd2aea88c0dd40d6a319ef2d7904a9db"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "054b08930200cfa1b353176683a765be"
server_key_iv: "f67c0cc4e5dcb63036f4a2a7f016adcc"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.445077 6804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:39.466984 6804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:39.467805 6804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:39.468027 6804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:39.468674 6804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:39.469741 6804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:39.469813 6804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.469887 6804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:39.469935 6804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.480108 6804 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:44939
I20260811 18:50:39.480139 6948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:44939 every 8 connection(s)
I20260811 18:50:39.481176 6804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:39.490242 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6804
I20260811 18:50:39.490358 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:39.490842 28111 external_mini_cluster.cc:1428] Setting key 2f6122b9282ae58b99793d4ca98d4f94
I20260811 18:50:39.492437 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.483450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:55275 (local address 127.27.115.254:35667)
0811 18:50:39.483627 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:39.483632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:39.484898 (+ 1266us) server_negotiation.cc:407] Connection header received
0811 18:50:39.486878 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:39.486881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:39.486983 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:39.487104 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:39.488206 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.488764 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.490039 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.490429 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.490560 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.491904 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:39.491909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:39.491911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:39.491926 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:39.491956 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:39.491965 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:39.492048 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:39.492222 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 18:50:39.492291 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:50:39.493433 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.483938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35667 (local address 127.27.115.195:55275)
0811 18:50:39.484694 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:39.484740 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:39.486638 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:39.487357 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:39.487365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:39.487482 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:39.488032 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.488046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.488917 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.488920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.489883 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:39.489895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:39.490638 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:39.490643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:39.490830 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:39.491603 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:39.491605 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:39.491652 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:39.491730 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:39.492049 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:39.492064 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:39.492071 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:39.492222 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 18:50:39.492561 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":138,"threads_started":1}
I20260811 18:50:39.494752 6949 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35667
I20260811 18:50:39.495036 6949 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:39.495631 6949 heartbeater.cc:511] Master 127.27.115.254:35667 requested a full tablet report, sending...
I20260811 18:50:39.496923 6439 ts_manager.cc:194] Registered new tserver with Master: cd2aea88c0dd40d6a319ef2d7904a9db (127.27.115.195:44939)
I20260811 18:50:39.497598 6439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.195:55275
I20260811 18:50:39.505769 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:50:39.509613 6488 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:39.508404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.27.115.254:35667)
0811 18:50:39.508588 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:39.508596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:39.508808 (+ 212us) server_negotiation.cc:407] Connection header received
0811 18:50:39.508979 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:39.508988 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:39.509052 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:39.509157 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:39.509449 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50576: BlockingRecv error: recv got EOF from 127.0.0.1:50576 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:50:39.510226 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6509
I20260811 18:50:39.522330 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6655
I20260811 18:50:39.531067 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6804
I20260811 18:50:39.539717 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6403
2026-08-11T18:50:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1005 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T18:50:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:39Z Disabled control of system clock
I20260811 18:50:39.579679 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36729
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:50:39.705777 6961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:39.706072 6961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:39.706110 6961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:39.710413 6961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:39.710487 6961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:39.710503 6961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:39.710553 6961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:39.710587 6961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:39.715564 6961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:36729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:39.716813 6961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:39.718361 6961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:39.727628 6968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:39.727628 6966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:39.727850 6961 server_base.cc:1034] running on GCE node
W20260811 18:50:39.727648 6970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:39.728715 6961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:39.729988 6961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:39.731235 6961 hybrid_clock.cc:648] HybridClock initialized: now 1786474239731214 us; error 39 us; skew 500 ppm
I20260811 18:50:39.733503 6961 webserver.cc:460] Webserver started at http://127.27.115.254:42597/ using document root <none> and password file <none>
I20260811 18:50:39.734230 6961 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:39.734335 6961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:39.734598 6961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:39.736892 6961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "00ab2dff7dd64227a17bb14526aeafac"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "387ced949ac68f93bb5cf8f61f75d088"
server_key_iv: "897e727cb1d62686b29334491624d14a"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.737519 6961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "00ab2dff7dd64227a17bb14526aeafac"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "387ced949ac68f93bb5cf8f61f75d088"
server_key_iv: "897e727cb1d62686b29334491624d14a"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.741704 6961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:39.744561 6979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:39.745970 6961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:50:39.746214 6961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "00ab2dff7dd64227a17bb14526aeafac"
format_stamp: "Formatted at 2026-08-11 18:50:39 on dist-test-slave-9jjr"
server_key: "387ced949ac68f93bb5cf8f61f75d088"
server_key_iv: "897e727cb1d62686b29334491624d14a"
server_key_version: "encryptionkey@0"
I20260811 18:50:39.746362 6961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:39.751979 6961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:39.753014 6961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:39.753257 6961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:39.761706 6961 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:36729
I20260811 18:50:39.761731 7035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:36729 every 8 connection(s)
I20260811 18:50:39.762872 6961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:39.766665 7036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:39.769480 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 6961
I20260811 18:50:39.769593 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:39.769922 28111 external_mini_cluster.cc:1428] Setting key 1256c7beb0eca5b99176d2dc355ffaa2
I20260811 18:50:39.773217 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac: Bootstrap starting.
I20260811 18:50:39.775831 7036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:39.775929 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:39.771576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.27.115.254:36729)
0811 18:50:39.772293 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:39.772309 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:39.772346 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:50:39.773017 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:39.773034 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:39.773448 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:39.773803 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:39.774608 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:39.774613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:39.774628 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:39.774646 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:39.774704 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:39.774719 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:39.774883 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:39.774957 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 18:50:39.775211 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":149,"threads_started":1}
I20260811 18:50:39.776854 7036 log.cc:824] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:39.779143 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac: No bootstrap required, opened a new log
I20260811 18:50:39.783006 7036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } }
I20260811 18:50:39.783278 7036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:39.783314 7036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ab2dff7dd64227a17bb14526aeafac, State: Initialized, Role: FOLLOWER
I20260811 18:50:39.783700 7036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } }
I20260811 18:50:39.783811 7036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:39.783859 7036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:39.783931 7036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:39.784868 7036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } }
I20260811 18:50:39.785181 7036 leader_election.cc:302] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ab2dff7dd64227a17bb14526aeafac; no voters:
I20260811 18:50:39.785458 7036 leader_election.cc:288] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:39.785597 7041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:39.785849 7041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [term 1 LEADER]: Becoming Leader. State: Replica: 00ab2dff7dd64227a17bb14526aeafac, State: Running, Role: LEADER
I20260811 18:50:39.786260 7041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } }
I20260811 18:50:39.786908 7036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:39.788076 7043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ab2dff7dd64227a17bb14526aeafac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } } }
I20260811 18:50:39.788213 7043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:39.788466 7044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ab2dff7dd64227a17bb14526aeafac. Latest consensus state: current_term: 1 leader_uuid: "00ab2dff7dd64227a17bb14526aeafac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab2dff7dd64227a17bb14526aeafac" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36729 } } }
I20260811 18:50:39.788632 7044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:39.788630 7052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:39.792186 7052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:39.798274 7052 catalog_manager.cc:1347] Generated new cluster ID: 6a420db1c63543118751bcc643b6fbb0
I20260811 18:50:39.798377 7052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:39.837069 7052 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:39.837551 7052 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:39.851569 7052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ab2dff7dd64227a17bb14526aeafac: Generated new TSK 0
I20260811 18:50:39.852569 7052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:39.856179 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--builtin_ntp_servers=127.27.115.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:50:39.983230 7062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:39.983546 7062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:39.983590 7062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:39.987965 7062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:39.988045 7062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:39.988108 7062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:39.993449 7062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:39.994709 7062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:39.996248 7062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.007728 7067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.007741 7068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.007807 7062 server_base.cc:1034] running on GCE node
W20260811 18:50:40.007625 7070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.008683 7062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.009430 7062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.010635 7062 hybrid_clock.cc:648] HybridClock initialized: now 1786474240010596 us; error 56 us; skew 500 ppm
I20260811 18:50:40.012895 7062 webserver.cc:460] Webserver started at http://127.27.115.193:39435/ using document root <none> and password file <none>
I20260811 18:50:40.013554 7062 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.013657 7062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.013913 7062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.016189 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "ef2051181652452496e8636f48c16cb1"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "aff472caa082430917e073fed1cdf66a"
server_key_iv: "4b993640c1b21ef3575bc2a4c1ea479f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.016829 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "ef2051181652452496e8636f48c16cb1"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "aff472caa082430917e073fed1cdf66a"
server_key_iv: "4b993640c1b21ef3575bc2a4c1ea479f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.021804 7062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:50:40.025959 7076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.027645 7062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:40.027858 7062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "ef2051181652452496e8636f48c16cb1"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "aff472caa082430917e073fed1cdf66a"
server_key_iv: "4b993640c1b21ef3575bc2a4c1ea479f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.028030 7062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.033922 7062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.034804 7062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.035050 7062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.035748 7062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.036876 7062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.036960 7062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.037041 7062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.037089 7062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.047794 7062 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:44375
I20260811 18:50:40.047806 7201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:44375 every 8 connection(s)
I20260811 18:50:40.048820 7062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:40.054804 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7062
I20260811 18:50:40.054934 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:40.055305 28111 external_mini_cluster.cc:1428] Setting key 85de58e08aa869233dca59d4fbe7dc40
I20260811 18:50:40.055680 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.050911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56739 (local address 127.27.115.254:36729)
0811 18:50:40.051062 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.051066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.052251 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 18:50:40.053723 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.053727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.053776 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.053868 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.055189 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.055193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.055196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:40.055209 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.055231 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:40.055239 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.055336 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.055522 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.055568 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:40.056700 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.051307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36729 (local address 127.27.115.193:56739)
0811 18:50:40.052067 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.052109 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.053479 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.054057 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.054065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.054222 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.054853 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:40.054856 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.054921 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:40.055016 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.055344 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.055355 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.055361 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.055555 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.055860 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":120,"threads_started":1}
I20260811 18:50:40.057554 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--builtin_ntp_servers=127.27.115.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:50:40.058085 7202 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36729
I20260811 18:50:40.058403 7202 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.058874 7202 heartbeater.cc:511] Master 127.27.115.254:36729 requested a full tablet report, sending...
I20260811 18:50:40.060675 6997 ts_manager.cc:194] Registered new tserver with Master: ef2051181652452496e8636f48c16cb1 (127.27.115.193:44375)
W20260811 18:50:40.194765 7206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.195070 7206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.195112 7206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.199349 7206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.199429 7206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.199491 7206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:40.204501 7206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:40.205698 7206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.207279 7206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.219264 7211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.219264 7215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.219318 7212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.219580 7206 server_base.cc:1034] running on GCE node
I20260811 18:50:40.220224 7206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.220973 7206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.222292 7206 hybrid_clock.cc:648] HybridClock initialized: now 1786474240222239 us; error 58 us; skew 500 ppm
I20260811 18:50:40.228714 7206 webserver.cc:460] Webserver started at http://127.27.115.194:37297/ using document root <none> and password file <none>
I20260811 18:50:40.229435 7206 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.229545 7206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.229812 7206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.232175 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "0d74ad0ab2fb41b69de2d77674a78ffa"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "5e2cd3a2f1eac2ec6083f0dca57f2f03"
server_key_iv: "be94664d83c678ae9d01537ef88623c3"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.233156 7206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "0d74ad0ab2fb41b69de2d77674a78ffa"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "5e2cd3a2f1eac2ec6083f0dca57f2f03"
server_key_iv: "be94664d83c678ae9d01537ef88623c3"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.237975 7206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:50:40.241483 7221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.243166 7206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:50:40.243363 7206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "0d74ad0ab2fb41b69de2d77674a78ffa"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "5e2cd3a2f1eac2ec6083f0dca57f2f03"
server_key_iv: "be94664d83c678ae9d01537ef88623c3"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.243523 7206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.248752 7206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.249497 7206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.249709 7206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.250509 7206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.251894 7206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.251950 7206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.251994 7206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.252009 7206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.262712 7206 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:42585
I20260811 18:50:40.262737 7346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:42585 every 8 connection(s)
I20260811 18:50:40.263813 7206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:40.266855 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7206
I20260811 18:50:40.266984 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:40.267318 28111 external_mini_cluster.cc:1428] Setting key 7406f988dbc0e8c64aa9daf68f550529
I20260811 18:50:40.269483 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--builtin_ntp_servers=127.27.115.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:50:40.271324 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.265955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:47705 (local address 127.27.115.254:36729)
0811 18:50:40.266109 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.266113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.267729 (+ 1616us) server_negotiation.cc:407] Connection header received
0811 18:50:40.269362 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.269366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.269412 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.269494 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.270814 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.270818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.270820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:40.270834 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.270856 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:40.270873 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.270976 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.271122 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.271181 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:50:40.272328 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.266406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36729 (local address 127.27.115.194:47705)
0811 18:50:40.267488 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.267571 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.269132 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.269718 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.269730 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.269840 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.270525 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:40.270527 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.270571 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:40.270644 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.270958 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.270967 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.270974 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.271176 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.271440 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":166,"threads_started":1}
I20260811 18:50:40.273618 7347 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36729
I20260811 18:50:40.273906 7347 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.274431 7347 heartbeater.cc:511] Master 127.27.115.254:36729 requested a full tablet report, sending...
I20260811 18:50:40.275622 6997 ts_manager.cc:194] Registered new tserver with Master: 0d74ad0ab2fb41b69de2d77674a78ffa (127.27.115.194:42585)
W20260811 18:50:40.394749 7350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.395136 7350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.395217 7350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.399727 7350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.399835 7350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.399951 7350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:40.405555 7350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:40.407081 7350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.408710 7350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.419821 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.419842 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.419858 7350 server_base.cc:1034] running on GCE node
W20260811 18:50:40.420116 7358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.420632 7350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.421507 7350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.422689 7350 hybrid_clock.cc:648] HybridClock initialized: now 1786474240422674 us; error 34 us; skew 500 ppm
I20260811 18:50:40.424960 7350 webserver.cc:460] Webserver started at http://127.27.115.195:35671/ using document root <none> and password file <none>
I20260811 18:50:40.425671 7350 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.425756 7350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.426040 7350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.428633 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "5ab82837fe1d454bb6eaea1f0e9405e5"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "9ccbf73afbd4be5192ab731cdd1c2443"
server_key_iv: "27e4fd4032358ffd95b444641fb0fda1"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.429418 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab82837fe1d454bb6eaea1f0e9405e5"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "9ccbf73afbd4be5192ab731cdd1c2443"
server_key_iv: "27e4fd4032358ffd95b444641fb0fda1"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.434405 7350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:50:40.437703 7364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.439548 7350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:40.439702 7350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "5ab82837fe1d454bb6eaea1f0e9405e5"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-9jjr"
server_key: "9ccbf73afbd4be5192ab731cdd1c2443"
server_key_iv: "27e4fd4032358ffd95b444641fb0fda1"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.439831 7350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.445590 7350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.446465 7350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.446658 7350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.447405 7350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.448585 7350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.448660 7350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.448736 7350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.448799 7350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.459327 7350 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:39633
I20260811 18:50:40.459347 7493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:39633 every 8 connection(s)
I20260811 18:50:40.460340 7350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:40.467130 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.462392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:49251 (local address 127.27.115.254:36729)
0811 18:50:40.462548 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.462552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.463747 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 18:50:40.465292 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.465295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.465341 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.465427 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.466695 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.466698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.466700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:50:40.466713 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.466733 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:50:40.466741 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.466832 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.466972 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.467013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:50:40.468119 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.462827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36729 (local address 127.27.115.195:49251)
0811 18:50:40.463571 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.463617 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.464957 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.465606 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.465614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.465706 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.466407 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:50:40.466409 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.466453 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:50:40.466527 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.466838 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.466845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.466850 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.466996 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.467243 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":139,"threads_started":1}
I20260811 18:50:40.468616 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7350
I20260811 18:50:40.468767 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:40.469250 28111 external_mini_cluster.cc:1428] Setting key b6e1dd10d1fe947bb8815936f7360e69
I20260811 18:50:40.469485 7494 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36729
I20260811 18:50:40.469762 7494 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.470369 7494 heartbeater.cc:511] Master 127.27.115.254:36729 requested a full tablet report, sending...
I20260811 18:50:40.471647 6996 ts_manager.cc:194] Registered new tserver with Master: 5ab82837fe1d454bb6eaea1f0e9405e5 (127.27.115.195:39633)
I20260811 18:50:40.484387 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:50:40.488108 7039 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:40.486870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.27.115.254:36729)
0811 18:50:40.487179 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.487185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.487337 (+ 152us) server_negotiation.cc:407] Connection header received
0811 18:50:40.487513 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.487518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.487591 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.487699 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.487975 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48254: BlockingRecv error: recv got EOF from 127.0.0.1:48254 (error 108)
Metrics: {"server-negotiator.queue_time_us":193}
I20260811 18:50:40.488494 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7062
I20260811 18:50:40.501526 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7206
I20260811 18:50:40.513640 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7350
I20260811 18:50:40.522532 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 6961
2026-08-11T18:50:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (984 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:40 dist-test-slave-9jjr krb5kdc[7504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:40 dist-test-slave-9jjr krb5kdc[7504](info): set up 2 sockets
Aug 11 18:50:40 dist-test-slave-9jjr krb5kdc[7504](info): commencing operation
krb5kdc: starting...
W20260811 18:50:41.649250 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:41 dist-test-slave-9jjr krb5kdc[7504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:50:41.794710 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35535
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:41.915858 7518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:41.916165 7518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:41.916209 7518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:41.920408 7518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:41.920487 7518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:41.920507 7518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:41.920524 7518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:41.920540 7518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:41.926209 7518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:41.927896 7518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:41.929227 7518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:41.938526 7525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.938516 7527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.938516 7523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.939023 7518 server_base.cc:1034] running on GCE node
I20260811 18:50:41.939747 7518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:41.940941 7518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:41.942198 7518 hybrid_clock.cc:648] HybridClock initialized: now 1786474241942165 us; error 42 us; skew 500 ppm
Aug 11 18:50:41 dist-test-slave-9jjr krb5kdc[7504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:41.945441 7518 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:50:41.946561 7518 webserver.cc:460] Webserver started at http://127.27.115.254:42847/ using document root <none> and password file <none>
I20260811 18:50:41.947166 7518 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:41.947270 7518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:41.947523 7518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:41.949733 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "c59768fc332d499cadef8ed2b5831d0c"
format_stamp: "Formatted at 2026-08-11 18:50:41 on dist-test-slave-9jjr"
server_key: "ab563f501997615bf7874fc9d9a04940"
server_key_iv: "fecd3de9b6c8b6e35bca4c498b91b26d"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.950424 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "c59768fc332d499cadef8ed2b5831d0c"
format_stamp: "Formatted at 2026-08-11 18:50:41 on dist-test-slave-9jjr"
server_key: "ab563f501997615bf7874fc9d9a04940"
server_key_iv: "fecd3de9b6c8b6e35bca4c498b91b26d"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.954757 7518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:41.957459 7535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.958920 7518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:41.959095 7518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "c59768fc332d499cadef8ed2b5831d0c"
format_stamp: "Formatted at 2026-08-11 18:50:41 on dist-test-slave-9jjr"
server_key: "ab563f501997615bf7874fc9d9a04940"
server_key_iv: "fecd3de9b6c8b6e35bca4c498b91b26d"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.959237 7518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:41.987879 7518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:41.991940 7518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:41.992197 7518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.000250 7518 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:35535
I20260811 18:50:42.000341 7590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:35535 every 8 connection(s)
I20260811 18:50:42.001343 7518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:42.003499 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7518
I20260811 18:50:42.003638 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:50:42.003991 28111 external_mini_cluster.cc:1428] Setting key 817c157a33bd4b71ddad65e3f38a636a
I20260811 18:50:42.004926 7591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:42.011104 7591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: Bootstrap starting.
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:42.013702 7591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:42.014573 7591 log.cc:824] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.016887 7591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: No bootstrap required, opened a new log
I20260811 18:50:42.019040 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.005459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.27.115.254:35535)
0811 18:50:42.005980 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.005996 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.006033 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:50:42.006797 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.006814 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.007150 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.007501 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:42.008336 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.009331 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.010260 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.010729 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.010910 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.013649 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:42.013654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:42.013668 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:42.013699 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:42.015791 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.016278 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.016285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.016297 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.016419 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.016711 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.016714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.016716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.017197 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:42.017419 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.017744 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.018089 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":166,"threads_started":1}
I20260811 18:50:42.021402 7591 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } }
I20260811 18:50:42.021778 7591 raft_consensus.cc:374] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:42.021858 7591 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59768fc332d499cadef8ed2b5831d0c, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.022392 7591 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } }
I20260811 18:50:42.022527 7591 raft_consensus.cc:388] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:42.022575 7591 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:42.022650 7591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:42.023694 7591 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } }
I20260811 18:50:42.024070 7591 leader_election.cc:302] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c59768fc332d499cadef8ed2b5831d0c; no voters:
I20260811 18:50:42.024384 7591 leader_election.cc:288] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:42.024535 7596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:42.024803 7596 raft_consensus.cc:686] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [term 1 LEADER]: Becoming Leader. State: Replica: c59768fc332d499cadef8ed2b5831d0c, State: Running, Role: LEADER
I20260811 18:50:42.025166 7596 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } }
I20260811 18:50:42.025884 7591 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:42.027215 7597 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c59768fc332d499cadef8ed2b5831d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } } }
I20260811 18:50:42.027217 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59768fc332d499cadef8ed2b5831d0c. Latest consensus state: current_term: 1 leader_uuid: "c59768fc332d499cadef8ed2b5831d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59768fc332d499cadef8ed2b5831d0c" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35535 } } }
I20260811 18:50:42.027401 7597 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:42.027401 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:42.027843 7606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:42.030861 7606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:42.036741 7606 catalog_manager.cc:1347] Generated new cluster ID: c16baf1069724a1bab4c72cdaefbaa08
I20260811 18:50:42.036839 7606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:42.051458 7606 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:42.052547 7606 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:42.059767 7606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: Generated new TSK 0
I20260811 18:50:42.060549 7606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:50:42.098420 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:42.223202 7620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.223538 7620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.223619 7620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.228094 7620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.228211 7620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.228307 7620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:50:42.234828 7620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:42.236416 7620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.237661 7620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.249945 7625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.250111 7620 server_base.cc:1034] running on GCE node
W20260811 18:50:42.250013 7626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.250013 7628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.251137 7620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.252040 7620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.253275 7620 hybrid_clock.cc:648] HybridClock initialized: now 1786474242253275 us; error 106 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.256776 7620 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:50:42.258376 7620 webserver.cc:460] Webserver started at http://127.27.115.193:39143/ using document root <none> and password file <none>
I20260811 18:50:42.258980 7620 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.259047 7620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.259241 7620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:42.261448 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "adebabe843f6470894269b75d67d960d"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d961407584549e91124fa7f6c52cdece"
server_key_iv: "597e90025ecc307782306a01c06c1421"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.262018 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "adebabe843f6470894269b75d67d960d"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d961407584549e91124fa7f6c52cdece"
server_key_iv: "597e90025ecc307782306a01c06c1421"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.267066 7620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:50:42.270215 7635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.272147 7620 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:50:42.272305 7620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "adebabe843f6470894269b75d67d960d"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d961407584549e91124fa7f6c52cdece"
server_key_iv: "597e90025ecc307782306a01c06c1421"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.272598 7620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.291672 7620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.295490 7620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.295738 7620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.296885 7620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.297986 7620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:42.298067 7620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.298185 7620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:42.298245 7620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.309196 7620 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:38467
I20260811 18:50:42.309216 7768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:38467 every 8 connection(s)
I20260811 18:50:42.310390 7620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:42.317598 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7620
I20260811 18:50:42.317713 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:42.318043 28111 external_mini_cluster.cc:1428] Setting key f34b6a5fae7eb4bb38658ddcef06f4e4
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:42.327785 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.312553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:42393 (local address 127.27.115.254:35535)
0811 18:50:42.312719 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.312723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.313680 (+ 957us) server_negotiation.cc:407] Connection header received
0811 18:50:42.315263 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.315266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.315333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.315445 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:42.316749 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.317348 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.318805 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.319128 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.319265 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.322578 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:42.322584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:42.322589 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:42.322634 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:42.324640 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.325468 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.325475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.325480 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.325591 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.326112 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.326117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.326122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.326448 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:42.326569 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.327334 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.327511 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:50:42.328521 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.312964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35535 (local address 127.27.115.193:42393)
0811 18:50:42.313491 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.313538 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.315031 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.315640 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.315647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.316045 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:42.316587 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.316600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.317491 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.317500 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.318651 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.318668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.319316 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.319319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.319563 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.320346 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:42.320366 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:42.322320 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:42.324902 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.324909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.324926 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.325300 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.325777 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.325784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.325789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.325987 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.326610 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:42.326617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:42.327021 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.327214 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.327559 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":124,"threads_started":1}
I20260811 18:50:42.329969 7770 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35535
I20260811 18:50:42.330335 7770 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:50:42.331027 7770 heartbeater.cc:511] Master 127.27.115.254:35535 requested a full tablet report, sending...
I20260811 18:50:42.332731 7554 ts_manager.cc:194] Registered new tserver with Master: adebabe843f6470894269b75d67d960d (127.27.115.193:38467)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:50:42.334250 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:42393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:50:42.354451 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:42.475054 7775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.475360 7775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.475440 7775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.479681 7775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.479784 7775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.479876 7775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:50:42.485333 7775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:42.486609 7775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.487725 7775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.497982 7784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.497982 7782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.497982 7781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.498247 7775 server_base.cc:1034] running on GCE node
I20260811 18:50:42.498855 7775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.499583 7775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.500799 7775 hybrid_clock.cc:648] HybridClock initialized: now 1786474242500764 us; error 55 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.504020 7775 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:50:42.505046 7775 webserver.cc:460] Webserver started at http://127.27.115.194:35247/ using document root <none> and password file <none>
I20260811 18:50:42.505653 7775 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.505748 7775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.506002 7775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:42.508288 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "5d63eade71fe4cfe864b1ab853994054"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d66c8ed7cbfb967202a41c5f4c62e3d2"
server_key_iv: "9e33076295f0497670b5a7487108def2"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.508872 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "5d63eade71fe4cfe864b1ab853994054"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d66c8ed7cbfb967202a41c5f4c62e3d2"
server_key_iv: "9e33076295f0497670b5a7487108def2"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.512804 7775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:50:42.515342 7791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.516564 7775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:50:42.516707 7775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "5d63eade71fe4cfe864b1ab853994054"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "d66c8ed7cbfb967202a41c5f4c62e3d2"
server_key_iv: "9e33076295f0497670b5a7487108def2"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.516829 7775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.540576 7775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.544329 7775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.544569 7775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.545279 7775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.546380 7775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:42.546459 7775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.546537 7775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:42.546587 7775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.556947 7775 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:40401
I20260811 18:50:42.556977 7922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:40401 every 8 connection(s)
I20260811 18:50:42.557909 7775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:42.564420 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7775
I20260811 18:50:42.564548 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:42.564858 28111 external_mini_cluster.cc:1428] Setting key fc46a4fde1d1bc58288e36756648c9f8
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:42.573872 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.560014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:35225 (local address 127.27.115.254:35535)
0811 18:50:42.560166 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.560173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.561075 (+ 902us) server_negotiation.cc:407] Connection header received
0811 18:50:42.562680 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.562684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.562744 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.562827 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:42.564130 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.564742 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.566364 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.566675 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.566803 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.569682 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:42.569686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:42.569688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:42.569723 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:42.571322 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.572058 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.572061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.572062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.572141 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.572596 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.572598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.572600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.572793 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:42.572869 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.573630 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.573746 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:50:42.575276 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.560426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35535 (local address 127.27.115.194:35225)
0811 18:50:42.560917 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.560962 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.562453 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.563000 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.563007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.563435 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:42.563943 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.563957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.564881 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.564890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.566242 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.566256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.566856 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.566859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.567010 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.567669 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:42.567686 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:42.569489 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:42.571488 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.571499 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.571520 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.571948 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.572242 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.572249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.572253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.572485 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.572908 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:42.572919 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:42.573332 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.573596 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.574014 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:50:42.576728 7923 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35535
I20260811 18:50:42.577080 7923 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:42.577903 7923 heartbeater.cc:511] Master 127.27.115.254:35535 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:50:42.579310 7554 ts_manager.cc:194] Registered new tserver with Master: 5d63eade71fe4cfe864b1ab853994054 (127.27.115.194:40401)
I20260811 18:50:42.580129 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:35225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:50:42.599380 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:42.724738 7929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.725061 7929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.725109 7929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.729688 7929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.729776 7929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.729863 7929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:50:42.735627 7929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:50:42.736897 7929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.738055 7929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.748878 7934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.748844 7935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.749221 7929 server_base.cc:1034] running on GCE node
W20260811 18:50:42.748879 7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.749854 7929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.750653 7929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.751880 7929 hybrid_clock.cc:648] HybridClock initialized: now 1786474242751843 us; error 50 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.755249 7929 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:50:42.756363 7929 webserver.cc:460] Webserver started at http://127.27.115.195:46523/ using document root <none> and password file <none>
I20260811 18:50:42.757020 7929 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.757136 7929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.757429 7929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:42.760286 7929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "ceffec59dc3a44d7a2906c10ca4c8454"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "71893ab69647950b37840d6331b5c242"
server_key_iv: "d7ee72e4058e58bc7b8219b855a7743f"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.761071 7929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "ceffec59dc3a44d7a2906c10ca4c8454"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "71893ab69647950b37840d6331b5c242"
server_key_iv: "d7ee72e4058e58bc7b8219b855a7743f"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.766100 7929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:50:42.769232 7947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.771057 7929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:50:42.771219 7929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "ceffec59dc3a44d7a2906c10ca4c8454"
format_stamp: "Formatted at 2026-08-11 18:50:42 on dist-test-slave-9jjr"
server_key: "71893ab69647950b37840d6331b5c242"
server_key_iv: "d7ee72e4058e58bc7b8219b855a7743f"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.771344 7929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.802414 7929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.806262 7929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.806447 7929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.807052 7929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.807950 7929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:42.808043 7929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.808130 7929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:42.808182 7929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.818470 7929 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:45923
I20260811 18:50:42.818567 8081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:45923 every 8 connection(s)
I20260811 18:50:42.819602 7929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:42.828769 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 7929
I20260811 18:50:42.828989 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:42.829474 28111 external_mini_cluster.cc:1428] Setting key 5ba3109cbc6dbf211dae27491b9fe868
Aug 11 18:50:42 dist-test-slave-9jjr krb5kdc[7504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:50:42.842674 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.822490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:47019 (local address 127.27.115.254:35535)
0811 18:50:42.822739 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.822747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.824036 (+ 1289us) server_negotiation.cc:407] Connection header received
0811 18:50:42.826841 (+ 2805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.826846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.826943 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.827063 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:42.829520 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.830380 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.832404 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.832742 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.832820 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.836871 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:42.836877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:42.836880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:42.836922 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:42.839285 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.840295 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.840300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.840302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.840426 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.840962 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.840966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.840967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.841254 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:42.841346 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.842345 (+ 999us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.842531 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:50:42.843849 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.822932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35535 (local address 127.27.115.195:47019)
0811 18:50:42.823787 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.823870 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.826251 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.827415 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.827432 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.828160 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:42.829160 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.829189 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.830572 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.830576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.832214 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.832232 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.833047 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.833050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.833249 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.834063 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:42.834085 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:42.836529 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:42.839547 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.839558 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.839590 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.840125 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.840579 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:42.840583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:42.840585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:42.840834 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:42.841459 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:42.841468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:42.841934 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.842236 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.842681 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":142,"threads_started":1}
I20260811 18:50:42.845610 8082 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35535
I20260811 18:50:42.845979 8082 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:42.846741 8082 heartbeater.cc:511] Master 127.27.115.254:35535 requested a full tablet report, sending...
I20260811 18:50:42.848181 7554 ts_manager.cc:194] Registered new tserver with Master: ceffec59dc3a44d7a2906c10ca4c8454 (127.27.115.195:45923)
I20260811 18:50:42.848904 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:47019
I20260811 18:50:42.854194 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:42.867147 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.857445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.27.115.254:35535)
0811 18:50:42.857622 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.857628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.857644 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:42.857712 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.857715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.857759 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.857873 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:42.859209 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.859756 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.860681 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.860952 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.861078 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.862831 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:42.862835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:42.862837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:42.862890 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:42.864855 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.865433 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.865438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.865440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.865554 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:42.865895 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:42.865899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:42.865901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:42.866271 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:42.866448 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.866770 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.866949 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:50:42.871660 7554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:42.875560 7554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:42.896957 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.891434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.27.115.194:40401)
0811 18:50:42.891878 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.891886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.891905 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:50:42.891973 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.891977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.892114 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.892277 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:42.892563 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893720 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.895397 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896066 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.896229 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.896404 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.896621 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.896752 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":166,"threads_started":1}
I20260811 18:50:42.896984 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.889546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:45923 (local address 127.0.0.1:56808)
0811 18:50:42.890010 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.890073 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.890266 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.891527 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.891534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.891559 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:42.891740 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.891748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893575 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.893579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.895335 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.895349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896491 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.896495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.896590 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.896623 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.896680 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.896785 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":175,"thread_start_us":135,"threads_started":1}
I20260811 18:50:42.897203 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.890682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.27.115.193:38467)
0811 18:50:42.891286 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.891294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.891322 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:50:42.891690 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.891696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.891917 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.892029 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:42.892509 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893415 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.895314 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896232 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.896416 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.896587 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.896905 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.897026 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":116,"threads_started":1}
I20260811 18:50:42.897531 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.891215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:40401 (local address 127.0.0.1:34956)
0811 18:50:42.891490 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.891505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.891614 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.892263 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.892267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.892288 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:42.892426 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.892437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893892 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.893896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.895231 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.895247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896223 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.896227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.896362 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.896405 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.896581 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.896759 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":36,"thread_start_us":124,"threads_started":1}
I20260811 18:50:42.897677 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.889747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38467 (local address 127.0.0.1:40844)
0811 18:50:42.890894 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.890911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.891526 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.892208 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.892211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.892230 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:42.892323 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.892330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893555 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.893560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.895121 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.895136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896652 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.896658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.896743 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.896763 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.897449 (+ 686us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.897545 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":124,"thread_start_us":370,"threads_started":1}
I20260811 18:50:42.898507 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.889760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.27.115.195:45923)
0811 18:50:42.890882 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.890896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.890930 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:50:42.891028 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.891034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.891312 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.891924 (+ 612us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:42.892113 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.893399 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.895486 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.896329 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.897931 (+ 1602us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.898116 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.898214 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.898339 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":437,"threads_started":1}
I20260811 18:50:42.900954 7692 tablet_service.cc:1467] Processing CreateTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 (DEFAULT_TABLE table=test-table [id=8e473e9b6b4940b09e648d99a7cf1950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:42.901064 7849 tablet_service.cc:1467] Processing CreateTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 (DEFAULT_TABLE table=test-table [id=8e473e9b6b4940b09e648d99a7cf1950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:42.901664 8004 tablet_service.cc:1467] Processing CreateTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 (DEFAULT_TABLE table=test-table [id=8e473e9b6b4940b09e648d99a7cf1950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:42.902405 7849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47bd3bbfbc684ab59a231d68a10ded99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:42.902405 7692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47bd3bbfbc684ab59a231d68a10ded99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:42.902798 8004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47bd3bbfbc684ab59a231d68a10ded99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:42.910421 8104 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: Bootstrap starting.
I20260811 18:50:42.911556 8105 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: Bootstrap starting.
I20260811 18:50:42.913455 8104 tablet_bootstrap.cc:654] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:42.914286 8105 tablet_bootstrap.cc:654] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:42.914546 8104 log.cc:824] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.914636 8107 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Bootstrap starting.
I20260811 18:50:42.915175 8105 log.cc:824] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.916923 8104 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: No bootstrap required, opened a new log
I20260811 18:50:42.917174 8104 ts_tablet_manager.cc:1397] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:50:42.917425 8105 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: No bootstrap required, opened a new log
I20260811 18:50:42.917624 8107 tablet_bootstrap.cc:654] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:42.917649 8105 ts_tablet_manager.cc:1397] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:50:42.918614 8107 log.cc:824] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.921330 8107 tablet_bootstrap.cc:492] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: No bootstrap required, opened a new log
I20260811 18:50:42.921581 8107 ts_tablet_manager.cc:1397] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:50:42.921799 8104 raft_consensus.cc:348] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.922077 8104 raft_consensus.cc:374] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:42.921904 8105 raft_consensus.cc:348] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.922164 8104 raft_consensus.cc:729] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adebabe843f6470894269b75d67d960d, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.922197 8105 raft_consensus.cc:374] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:42.922251 8105 raft_consensus.cc:729] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d63eade71fe4cfe864b1ab853994054, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.922670 8104 consensus_queue.cc:260] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.922721 8105 consensus_queue.cc:260] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.923465 7770 heartbeater.cc:503] Master 127.27.115.254:35535 was elected leader, sending a full tablet report...
I20260811 18:50:42.923811 8104 ts_tablet_manager.cc:1428] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260811 18:50:42.923843 8105 ts_tablet_manager.cc:1428] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:50:42.924209 7923 heartbeater.cc:503] Master 127.27.115.254:35535 was elected leader, sending a full tablet report...
I20260811 18:50:42.927305 8107 raft_consensus.cc:348] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.928257 8107 raft_consensus.cc:374] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:42.928340 8107 raft_consensus.cc:729] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceffec59dc3a44d7a2906c10ca4c8454, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.928972 8107 consensus_queue.cc:260] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.929872 8082 heartbeater.cc:503] Master 127.27.115.254:35535 was elected leader, sending a full tablet report...
I20260811 18:50:42.930377 8107 ts_tablet_manager.cc:1428] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:50:42.942036 8113 raft_consensus.cc:482] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:42.942353 8113 raft_consensus.cc:504] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.943642 8113 leader_election.cc:288] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceffec59dc3a44d7a2906c10ca4c8454 (127.27.115.195:45923), adebabe843f6470894269b75d67d960d (127.27.115.193:38467)
I20260811 18:50:42.947901 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.943928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:45923 (local address 127.27.115.194:43229)
0811 18:50:42.944094 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.944110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.944214 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.944986 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.944989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.945006 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:42.945136 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.945141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.945859 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.945864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.946766 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.946777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.947452 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.947456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.947561 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.947585 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.947675 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.947756 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":2}
I20260811 18:50:42.947899 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.944030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:43229 (local address 127.27.115.195:45923)
0811 18:50:42.944182 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.944188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.944207 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:50:42.944801 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.944804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.944870 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.944939 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:42.945252 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.945745 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.946895 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.947306 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.947467 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.947525 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.947691 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.947763 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:50:42.948474 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.944051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:47601 (local address 127.27.115.193:38467)
0811 18:50:42.944192 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.944199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.944625 (+ 426us) server_negotiation.cc:407] Connection header received
0811 18:50:42.944754 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.944757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.944807 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.944920 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:42.945213 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.945732 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.946918 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.947354 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.947836 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.948036 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.948267 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.948359 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:50:42.948484 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.943935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38467 (local address 127.27.115.194:47601)
0811 18:50:42.944514 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.944529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.944649 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.944936 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:42.944939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:42.944957 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:42.945093 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.945098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.945853 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.945858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.946778 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:42.946788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.947864 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.947871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:42.948109 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.948138 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:50:42.948243 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:50:42.948316 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":191,"thread_start_us":156,"threads_started":1}
I20260811 18:50:42.948699 8027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47bd3bbfbc684ab59a231d68a10ded99" candidate_uuid: "5d63eade71fe4cfe864b1ab853994054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" is_pre_election: true
I20260811 18:50:42.949162 8027 raft_consensus.cc:2454] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d63eade71fe4cfe864b1ab853994054 in term 0.
I20260811 18:50:42.949134 7717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47bd3bbfbc684ab59a231d68a10ded99" candidate_uuid: "5d63eade71fe4cfe864b1ab853994054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adebabe843f6470894269b75d67d960d" is_pre_election: true
I20260811 18:50:42.949406 7717 raft_consensus.cc:2454] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d63eade71fe4cfe864b1ab853994054 in term 0.
I20260811 18:50:42.949761 7792 leader_election.cc:302] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d63eade71fe4cfe864b1ab853994054, ceffec59dc3a44d7a2906c10ca4c8454; no voters:
I20260811 18:50:42.950107 8113 raft_consensus.cc:2781] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:42.950258 8113 raft_consensus.cc:482] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:42.950337 8113 raft_consensus.cc:3037] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:42.951617 8113 raft_consensus.cc:504] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.952060 8113 leader_election.cc:288] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [CANDIDATE]: Term 1 election: Requested vote from peers ceffec59dc3a44d7a2906c10ca4c8454 (127.27.115.195:45923), adebabe843f6470894269b75d67d960d (127.27.115.193:38467)
I20260811 18:50:42.952605 8027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47bd3bbfbc684ab59a231d68a10ded99" candidate_uuid: "5d63eade71fe4cfe864b1ab853994054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceffec59dc3a44d7a2906c10ca4c8454"
I20260811 18:50:42.952764 8027 raft_consensus.cc:3037] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:42.953092 7717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47bd3bbfbc684ab59a231d68a10ded99" candidate_uuid: "5d63eade71fe4cfe864b1ab853994054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adebabe843f6470894269b75d67d960d"
I20260811 18:50:42.953256 7717 raft_consensus.cc:3037] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:42.954205 8027 raft_consensus.cc:2454] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d63eade71fe4cfe864b1ab853994054 in term 1.
I20260811 18:50:42.954587 7792 leader_election.cc:302] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d63eade71fe4cfe864b1ab853994054, ceffec59dc3a44d7a2906c10ca4c8454; no voters:
I20260811 18:50:42.954617 7717 raft_consensus.cc:2454] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d63eade71fe4cfe864b1ab853994054 in term 1.
I20260811 18:50:42.954824 8113 raft_consensus.cc:2781] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:42.955092 8113 raft_consensus.cc:686] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 1 LEADER]: Becoming Leader. State: Replica: 5d63eade71fe4cfe864b1ab853994054, State: Running, Role: LEADER
I20260811 18:50:42.955499 8113 consensus_queue.cc:237] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } }
I20260811 18:50:42.959023 7553 catalog_manager.cc:5495] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d63eade71fe4cfe864b1ab853994054 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "5d63eade71fe4cfe864b1ab853994054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d63eade71fe4cfe864b1ab853994054" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40401 } health_report { overall_health: HEALTHY } } }
I20260811 18:50:42.979321 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.975359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.27.115.194:40401)
0811 18:50:42.975526 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:42.975531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:42.975547 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:42.975699 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:42.975701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:42.975826 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:42.975943 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:42.976211 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.976720 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.977615 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:42.977848 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:42.977957 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:42.978371 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:42.978853 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:42.978995 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:42.979064 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:50:42.979135 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:50:42.985798 8027 raft_consensus.cc:1261] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 1 FOLLOWER]: Refusing update from remote peer 5d63eade71fe4cfe864b1ab853994054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:42.985798 7717 raft_consensus.cc:1261] T 47bd3bbfbc684ab59a231d68a10ded99 P adebabe843f6470894269b75d67d960d [term 1 FOLLOWER]: Refusing update from remote peer 5d63eade71fe4cfe864b1ab853994054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:42.986685 8116 consensus_queue.cc:1035] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceffec59dc3a44d7a2906c10ca4c8454" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 45923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:42.986881 8113 consensus_queue.cc:1035] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adebabe843f6470894269b75d67d960d" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 38467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:42.995663 8122 mvcc.cc:205] Tried to move back new op lower bound from 7317398499261480960 to 7317398499149058048. Current Snapshot: MvccSnapshot[applied={T|T < 7317398499261480960}]
I20260811 18:50:42.996109 8120 mvcc.cc:205] Tried to move back new op lower bound from 7317398499261480960 to 7317398499149058048. Current Snapshot: MvccSnapshot[applied={T|T < 7317398499261480960}]
I20260811 18:50:43.005230 7554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:43.005396 7554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:43.007812 7554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c59768fc332d499cadef8ed2b5831d0c: Sending DeleteTablet for 3 replicas of tablet 47bd3bbfbc684ab59a231d68a10ded99
I20260811 18:50:43.008618 8004 tablet_service.cc:1514] Processing DeleteTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:56808
I20260811 18:50:43.008635 7692 tablet_service.cc:1514] Processing DeleteTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:40844
I20260811 18:50:43.009153 8132 tablet_replica.cc:330] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: stopping tablet replica
I20260811 18:50:43.009526 8132 raft_consensus.cc:2229] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:43.009557 7849 tablet_service.cc:1514] Processing DeleteTablet for tablet 47bd3bbfbc684ab59a231d68a10ded99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34956
I20260811 18:50:43.009899 8132 raft_consensus.cc:2258] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:43.011078 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7620
I20260811 18:50:43.015471 8132 ts_tablet_manager.cc:1905] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:43.019228 8132 ts_tablet_manager.cc:1918] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:43.019374 8132 log.cc:1196] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/47bd3bbfbc684ab59a231d68a10ded99
I20260811 18:50:43.019729 8132 ts_tablet_manager.cc:1939] T 47bd3bbfbc684ab59a231d68a10ded99 P ceffec59dc3a44d7a2906c10ca4c8454: Deleting consensus metadata
I20260811 18:50:43.020056 8134 tablet_replica.cc:330] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054: stopping tablet replica
I20260811 18:50:43.021314 7536 catalog_manager.cc:4841] TS ceffec59dc3a44d7a2906c10ca4c8454 (127.27.115.195:45923): tablet 47bd3bbfbc684ab59a231d68a10ded99 (table test-table [id=8e473e9b6b4940b09e648d99a7cf1950]) successfully deleted
W20260811 18:50:43.024865 7540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:38467 (error 108)
I20260811 18:50:43.025250 8134 raft_consensus.cc:2229] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:43.025818 8134 raft_consensus.cc:2258] T 47bd3bbfbc684ab59a231d68a10ded99 P 5d63eade71fe4cfe864b1ab853994054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:43.025836 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7775
I20260811 18:50:43.028568 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:43.025815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:38467 (local address 127.0.0.1:40846)
0811 18:50:43.025940 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:43.026022 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:50:43.028844 7540 catalog_manager.cc:4584] TS adebabe843f6470894269b75d67d960d (127.27.115.193:38467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47bd3bbfbc684ab59a231d68a10ded99: Network error: Client connection negotiation failed: client connection to 127.27.115.193:38467: connect: Connection refused (error 111)
I20260811 18:50:43.034407 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7929
I20260811 18:50:43.034997 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:43.034639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:40401 (local address 127.0.0.1:34988)
0811 18:50:43.034840 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:43.034898 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.194:40401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 18:50:43.044693 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 7518
2026-08-11T18:50:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2519 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:43 dist-test-slave-9jjr krb5kdc[8139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:43 dist-test-slave-9jjr krb5kdc[8139](info): set up 2 sockets
Aug 11 18:50:43 dist-test-slave-9jjr krb5kdc[8139](info): commencing operation
krb5kdc: starting...
W20260811 18:50:44.204381 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:50:44 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangeradmin_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangeradmin_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangerlookup_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangerlookup_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
I20260811 18:50:44.427336 28111 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4WGUPP/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/postgres -l logfile start
2026-08-11 18:50:47.740 UTC [8172] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:50:47.741 UTC [8172] LOG: listening on IPv4 address "127.27.115.212", port 34043
2026-08-11 18:50:47.745 UTC [8172] LOG: listening on Unix socket "/tmp/.s.PGSQL.34043"
2026-08-11 18:50:47.770 UTC [8175] LOG: database system was shut down at 2026-08-11 18:50:45 UTC
2026-08-11 18:50:47.776 UTC [8172] LOG: database system is ready to accept connections
I20260811 18:50:48.820149 28111 mini_postgres.cc:96] Postgres bound to 34043
2026-08-11 18:50:48.830 UTC [8185] FATAL: database "slave" does not exist
127.27.115.212:34043 - accepting connections
I20260811 18:50:48.831377 28111 mini_ranger.cc:161] Starting Ranger...
I20260811 18:50:48.850728 28111 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:50:49.487246 28111 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:50:49.487355 28111 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin
2026-08-11 18:50:49,814 [I] DB FLAVOR :POSTGRES
2026-08-11 18:50:49,814 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:50:49,815 [I] Checking connection..
2026-08-11 18:50:49,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:50:50,087 [I] Checking connection passed.
2026-08-11 18:50:50,088 [I] --------- Verifying version history table ---------
2026-08-11 18:50:50,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:50:50,384 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:50:50,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:50:50,648 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:50:50,648 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:50:50,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:50:50.947 UTC [8260] WARNING: there is no transaction in progress
2026-08-11 18:50:50,956 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:50:50,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:50:51,213 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:50:51,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:50:51,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:50:51,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:50:51,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9jjr.c.gcp-upstream.internal','N') ;"
2026-08-11 18:50:52,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:50:52,263 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:50:52,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:50:52,532 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:50:52,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:50:52,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:50:53,069 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:50:53,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:50:54.279 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.299 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.319 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.338 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.358 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.381 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.458 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.467 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.487 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.500 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.513 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.531 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.542 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.550 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.559 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:54.567 UTC [8386] WARNING: there is no transaction in progress
2026-08-11 18:50:55,476 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:50:55,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:50:55,748 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:50:55,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:50:56,016 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:50:56,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:50:56,296 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:50:56,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:50:56,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:50:56,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9jjr.c.gcp-upstream.internal';"
2026-08-11 18:50:57,100 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:50:57,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:50:57,366 [I] DB_PATCHES have already been applied
I20260811 18:50:57.372566 28111 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4WGUPP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:50:57.372645 28111 mini_ranger.cc:193] Using host: 127.27.115.212
I20260811 18:50:57.377581 28111 mini_ranger.cc:227] Ranger admin URL: http://127.27.115.212:43529
Aug 11, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:50:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:50:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:50:58.635406 28111 mini_ranger.cc:160] Time spent starting Ranger: real 9.804s user 0.001s sys 0.014s
Aug 11, 2026 6:51:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:51:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 6:51:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:51:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:51:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:51:25.420722 28111 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangerkms_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/rangerkms_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.115.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.115.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:51:25 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474285, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:51:25.538662 28111 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:51:25.552691 28111 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:51:26.965375 28111 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:51:26.965559 28111 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms
I20260811 18:51:26.965755 28111 mini_ranger_kms.cc:227] Using postgres at 127.27.115.212:34043
2026-08-11 18:51:27,240 [I] DB FLAVOR :POSTGRES
2026-08-11 18:51:27,240 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:51:27,240 [I] Checking connection
2026-08-11 18:51:27,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:51:27,515 [I] connection success
2026-08-11 18:51:27,515 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:51:27,516 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:51:27,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:51:27,789 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:51:27,790 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:51:27,790 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:51:27,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:34043/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:51:28,131 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:51:28.779933 28111 mini_ranger_kms.cc:310] Created kms service
I20260811 18:51:29.020380 28111 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:51:29.078802 28111 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:51:29 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474285, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:51:29.540102 28111 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:51:29.540271 28111 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms:/tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:51:29.540297 28111 mini_ranger_kms.cc:240] Using host: 127.27.115.212
I20260811 18:51:29.545037 28111 mini_ranger_kms.cc:276] Ranger KMS PID: 8597
I20260811 18:51:29.545212 28111 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.115.212:60167
Aug 11, 2026 6:51:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:51:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:51:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:51:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:51:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:51:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 18:51:32.186697 28111 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.648s user 0.000s sys 0.254s
I20260811 18:51:32.186866 28111 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:51:32.186986 28111 mini_ranger_kms.cc:401] 127.27.115.212:60167/kms/v1/keys
Aug 11 18:51:32 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474292, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 18:51:32 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474292, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
Aug 11 18:51:33 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474292, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
Aug 11 18:51:35 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:51:35.935071 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44759
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:44759
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:60167/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:36.063678 8657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:36.063994 8657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:36.064036 8657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:36.068583 8657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:51:36.068672 8657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:36.068693 8657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:36.068710 8657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:51:36.068728 8657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:51:36.075059 8657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.115.212:60167/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:44759
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:51:36.076387 8657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:36.077747 8657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:36.089174 8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:36.089254 8663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:36.089437 8657 server_base.cc:1034] running on GCE node
W20260811 18:51:36.089196 8662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:36.090832 8657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:36.092523 8657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:36.094302 8657 hybrid_clock.cc:648] HybridClock initialized: now 1786474296094268 us; error 62 us; skew 500 ppm
Aug 11 18:51:36 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474296, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:36.102923 8657 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:51:36.104264 8657 webserver.cc:460] Webserver started at http://127.27.115.254:45821/ using document root <none> and password file <none>
I20260811 18:51:36.104934 8657 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:36.105026 8657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:36.105273 8657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:51:36 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474296, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:51:36.179172 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7"
format_stamp: "Formatted at 2026-08-11 18:51:36 on dist-test-slave-9jjr"
server_key: "9594a35a983a04133ef99ecc1bf974f6"
server_key_iv: "dcdad8f3b969f1f3544678a209cb8ed9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:36.180395 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7"
format_stamp: "Formatted at 2026-08-11 18:51:36 on dist-test-slave-9jjr"
server_key: "9594a35a983a04133ef99ecc1bf974f6"
server_key_iv: "dcdad8f3b969f1f3544678a209cb8ed9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:36.185241 8657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 18:51:36.204846 8674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:36.206753 8657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:51:36.206956 8657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7"
format_stamp: "Formatted at 2026-08-11 18:51:36 on dist-test-slave-9jjr"
server_key: "9594a35a983a04133ef99ecc1bf974f6"
server_key_iv: "dcdad8f3b969f1f3544678a209cb8ed9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:36.207118 8657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:36.225025 8657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:36.228960 8657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:36.229461 8657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:36.238781 8657 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:44759
I20260811 18:51:36.238811 8728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:44759 every 8 connection(s)
I20260811 18:51:36.240046 8657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:51:36.244352 8729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:36.246428 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 8657
I20260811 18:51:36.246582 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:51:36.246886 28111 external_mini_cluster.cc:1428] Setting key bfbe8970b2102e3914d3b4e631d35edc
I20260811 18:51:36.252141 8729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: Bootstrap starting.
I20260811 18:51:36.254925 8729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:36.255839 8729 log.cc:824] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:51:36 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:51:36.258742 8729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: No bootstrap required, opened a new log
I20260811 18:51:36.262051 8729 raft_consensus.cc:348] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } }
I20260811 18:51:36.262444 8729 raft_consensus.cc:374] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:36.262532 8729 raft_consensus.cc:729] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d11ce1cd11dc497ea4c0dd66e32a4ae7, State: Initialized, Role: FOLLOWER
I20260811 18:51:36.263167 8729 consensus_queue.cc:260] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } }
I20260811 18:51:36.263361 8729 raft_consensus.cc:388] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:51:36.263435 8729 raft_consensus.cc:482] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:51:36.263585 8729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:36.264462 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:36.248568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.27.115.254:44759)
0811 18:51:36.249291 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:36.249309 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:36.249356 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:51:36.250195 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:36.250232 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:36.250656 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:36.251047 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:36.251964 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:36.253062 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:36.254208 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:36.254774 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:36.254978 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:36.258391 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:36.258398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:36.258414 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:36.258447 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:36.260972 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:36.261541 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:36.261547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:36.261553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:36.261678 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:36.261988 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:36.261992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:36.261994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:36.262571 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:36.262802 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:36.263202 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 18:51:36.263566 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":195,"threads_started":1}
I20260811 18:51:36.264886 8729 raft_consensus.cc:504] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } }
I20260811 18:51:36.265269 8729 leader_election.cc:302] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d11ce1cd11dc497ea4c0dd66e32a4ae7; no voters:
I20260811 18:51:36.265709 8729 leader_election.cc:288] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:51:36.265800 8734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:36.266027 8734 raft_consensus.cc:686] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [term 1 LEADER]: Becoming Leader. State: Replica: d11ce1cd11dc497ea4c0dd66e32a4ae7, State: Running, Role: LEADER
I20260811 18:51:36.266439 8734 consensus_queue.cc:237] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } }
I20260811 18:51:36.267146 8729 sys_catalog.cc:564] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:51:36.268461 8736 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d11ce1cd11dc497ea4c0dd66e32a4ae7. Latest consensus state: current_term: 1 leader_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } } }
I20260811 18:51:36.268471 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11ce1cd11dc497ea4c0dd66e32a4ae7" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44759 } } }
I20260811 18:51:36.268734 8736 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:36.268767 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:36.269389 8747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:51:36.269943 8729 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:51:36.276409 8747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:51:36.285921 8747 catalog_manager.cc:1347] Generated new cluster ID: f68b2350bcab43bb9b7328dc4f93977c
I20260811 18:51:36.286079 8747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:51:36.299239 8747 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:51:36.300555 8747 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:51:36.306850 8747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: Generated new TSK 0
I20260811 18:51:36.307751 8747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:51:37.083132 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:60167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:37.258608 8780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:37.259141 8780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:37.259259 8780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:37.266166 8780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:37.266345 8780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:37.266464 8780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:51:37.275414 8780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.115.212:60167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:51:37.277154 8780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:37.278963 8780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:37.294075 8790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:37.294296 8792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:37.294082 8788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:37.294790 8780 server_base.cc:1034] running on GCE node
I20260811 18:51:37.295378 8780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:37.296196 8780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:37.297446 8780 hybrid_clock.cc:648] HybridClock initialized: now 1786474297297410 us; error 58 us; skew 500 ppm
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:37.301640 8780 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:51:37.302949 8780 webserver.cc:460] Webserver started at http://127.27.115.193:32887/ using document root <none> and password file <none>
I20260811 18:51:37.303727 8780 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:37.303858 8780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:37.304136 8780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:51:37.324738 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "118bb82d02f24867b8fd7f98d48d21fa"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "58bb88dfe13b6fc9a5537685dc7f1715"
server_key_iv: "54588ef1a933ba616ab1ae36b20268cd"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.325820 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "118bb82d02f24867b8fd7f98d48d21fa"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "58bb88dfe13b6fc9a5537685dc7f1715"
server_key_iv: "54588ef1a933ba616ab1ae36b20268cd"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.352087 8780 fs_manager.cc:691] Time spent creating directory manager: real 0.026s user 0.002s sys 0.003s
I20260811 18:51:37.372015 8803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.373641 8780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:51:37.373816 8780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "118bb82d02f24867b8fd7f98d48d21fa"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "58bb88dfe13b6fc9a5537685dc7f1715"
server_key_iv: "54588ef1a933ba616ab1ae36b20268cd"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.374061 8780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:37.393111 8780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:37.397358 8780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:37.397660 8780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:37.398581 8780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:37.399905 8780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:37.400022 8780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.400154 8780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:37.400254 8780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.416024 8780 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:39653
I20260811 18:51:37.417573 8780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:51:37.423153 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 8780
I20260811 18:51:37.423306 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:51:37.423623 28111 external_mini_cluster.cc:1428] Setting key 7291a2f5cb1145e38f795caff6553d3f
I20260811 18:51:37.419950 8935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:39653 every 8 connection(s)
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:51:37.448879 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:37.422594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44759 (local address 127.27.115.193:43163)
0811 18:51:37.423261 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:37.423330 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:37.424471 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:37.431071 (+ 6600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:37.431085 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:37.431532 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:37.432366 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:37.432393 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.433886 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.433891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:37.435473 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:37.435493 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.436320 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.436325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:37.436597 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:37.437475 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:37.437498 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:37.442251 (+ 4753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:37.444861 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:37.444870 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:37.444893 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:37.445354 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:37.445738 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:37.445741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:37.445743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:37.445955 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:37.446536 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:37.446544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:37.446940 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 18:51:37.447191 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 18:51:37.447649 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":170,"threads_started":1}
I20260811 18:51:37.449898 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:37.421980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:43163 (local address 127.27.115.254:44759)
0811 18:51:37.430416 (+ 8436us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:37.430422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:37.430448 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:51:37.430507 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:37.430511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:37.430572 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:37.432638 (+ 2066us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:37.432778 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.433622 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.435722 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.436087 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.437060 (+ 973us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:37.442589 (+ 5529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:37.442594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:37.442598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:37.442634 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:37.444660 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:37.445499 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:37.445503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:37.445505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:37.445624 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:37.446073 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:37.446077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:37.446078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:37.446373 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:37.446475 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:37.449554 (+ 3079us) server_negotiation.cc:299] Negotiation successful
0811 18:51:37.449728 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8309,"thread_start_us":130,"threads_started":1}
I20260811 18:51:37.451401 8936 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44759
I20260811 18:51:37.451699 8936 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:37.452401 8936 heartbeater.cc:511] Master 127.27.115.254:44759 requested a full tablet report, sending...
I20260811 18:51:37.454595 8692 ts_manager.cc:194] Registered new tserver with Master: 118bb82d02f24867b8fd7f98d48d21fa (127.27.115.193:39653)
I20260811 18:51:37.456228 8692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:43163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:51:37.474679 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:60167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:51:37.663859 8943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:37.664393 8943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:37.664530 8943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:37.671557 8943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:37.671748 8943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:37.671927 8943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:51:37.680967 8943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.115.212:60167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:51:37.682945 8943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:37.684590 8943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:37.698763 8951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:37.699429 8954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:37.699952 8949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:37.700891 8943 server_base.cc:1034] running on GCE node
I20260811 18:51:37.701478 8943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:37.702486 8943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:37.703711 8943 hybrid_clock.cc:648] HybridClock initialized: now 1786474297703681 us; error 52 us; skew 500 ppm
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:37.708191 8943 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:51:37.709704 8943 webserver.cc:460] Webserver started at http://127.27.115.194:35291/ using document root <none> and password file <none>
I20260811 18:51:37.710664 8943 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:37.710909 8943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:37.711220 8943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:51:37.739918 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "674279fd4402452184a504a64a97baae"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "1af5bffc7580a1f6fed4769e262d3230"
server_key_iv: "5aaaa6c73ce3951227b958fd9be27bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.740859 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "674279fd4402452184a504a64a97baae"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "1af5bffc7580a1f6fed4769e262d3230"
server_key_iv: "5aaaa6c73ce3951227b958fd9be27bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.745788 8943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:51:37.772816 8965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.775367 8943 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260811 18:51:37.775597 8943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "674279fd4402452184a504a64a97baae"
format_stamp: "Formatted at 2026-08-11 18:51:37 on dist-test-slave-9jjr"
server_key: "1af5bffc7580a1f6fed4769e262d3230"
server_key_iv: "5aaaa6c73ce3951227b958fd9be27bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:37.775816 8943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:37.804340 8943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:37.809327 8943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:37.809660 8943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:37.810678 8943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:37.812225 8943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:37.812417 8943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.812549 8943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:37.812729 8943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:37.834154 8943 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:45051
I20260811 18:51:37.834362 9099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:45051 every 8 connection(s)
I20260811 18:51:37.835492 8943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:51:37.843495 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 8943
I20260811 18:51:37.843643 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:51:37.843988 28111 external_mini_cluster.cc:1428] Setting key 30df95d65faa8bdcd4fe5cb40c07181a
Aug 11 18:51:37 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:51:37.854434 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:37.838510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44759 (local address 127.27.115.194:48825)
0811 18:51:37.839073 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:37.839134 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:37.840104 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:37.840737 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:37.840744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:37.841052 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:37.841562 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:37.841575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.842497 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.842500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:37.843654 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:37.843668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.844499 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.844504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:37.844723 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:37.845660 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:37.845685 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:37.847551 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:37.850232 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:37.850242 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:37.850266 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:37.850699 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:37.851117 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:37.851123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:37.851127 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:37.851365 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:37.851972 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:37.851983 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:37.852377 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 18:51:37.852604 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:51:37.853075 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":170,"threads_started":1}
I20260811 18:51:37.855430 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:37.838008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:48825 (local address 127.27.115.254:44759)
0811 18:51:37.838206 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:37.838211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:37.839256 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 18:51:37.840363 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:37.840367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:37.840434 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:37.840538 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:37.841734 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.842363 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.843832 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:37.844170 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:37.844256 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:37.847815 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:37.847820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:37.847823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:37.847883 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:37.850017 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:37.850823 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:37.850830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:37.850835 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:37.850978 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:37.851493 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:37.851499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:37.851504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:37.851797 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:37.851907 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:37.855020 (+ 3113us) server_negotiation.cc:299] Negotiation successful
0811 18:51:37.855192 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:51:37.856631 9100 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44759
I20260811 18:51:37.856948 9100 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:37.857661 9100 heartbeater.cc:511] Master 127.27.115.254:44759 requested a full tablet report, sending...
I20260811 18:51:37.859210 8692 ts_manager.cc:194] Registered new tserver with Master: 674279fd4402452184a504a64a97baae (127.27.115.194:45051)
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:51:37.860132 8692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:48825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:51:37.891292 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:60167/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:38.082280 9105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:38.082799 9105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:38.083003 9105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:38.094434 9105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:38.094615 9105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:38.094779 9105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:51:38.103943 9105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.115.212:60167/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:51:38.105803 9105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:38.107491 9105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:38.125968 9111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:38.125973 9110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:38.126643 9113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:38.126959 9105 server_base.cc:1034] running on GCE node
I20260811 18:51:38.127579 9105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:38.128522 9105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:38.129824 9105 hybrid_clock.cc:648] HybridClock initialized: now 1786474298129753 us; error 85 us; skew 500 ppm
Aug 11 18:51:38 dist-test-slave-9jjr krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474298, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:38.135165 9105 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:51:38.136636 9105 webserver.cc:460] Webserver started at http://127.27.115.195:42729/ using document root <none> and password file <none>
I20260811 18:51:38.137543 9105 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:38.137722 9105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:38.138077 9105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:51:38 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474298, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
Aug 11 18:51:38 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:51:38.163693 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "7e27e27b9683465495da4ec55fab529d"
format_stamp: "Formatted at 2026-08-11 18:51:38 on dist-test-slave-9jjr"
server_key: "166b6e8c011d6e73b75d18e4eabc10d8"
server_key_iv: "dfbbb913d8e65f6b0bbd9582e5af056d"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:38.164847 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "7e27e27b9683465495da4ec55fab529d"
format_stamp: "Formatted at 2026-08-11 18:51:38 on dist-test-slave-9jjr"
server_key: "166b6e8c011d6e73b75d18e4eabc10d8"
server_key_iv: "dfbbb913d8e65f6b0bbd9582e5af056d"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:38.171744 9105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 18:51:38.196033 9122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:38.197602 9105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:51:38.197778 9105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "7e27e27b9683465495da4ec55fab529d"
format_stamp: "Formatted at 2026-08-11 18:51:38 on dist-test-slave-9jjr"
server_key: "166b6e8c011d6e73b75d18e4eabc10d8"
server_key_iv: "dfbbb913d8e65f6b0bbd9582e5af056d"
server_key_version: "kuduclusterkey@0"
I20260811 18:51:38.197944 9105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:38.225677 9105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:38.230180 9105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:38.230521 9105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:38.231441 9105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:38.232812 9105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:38.232937 9105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:38.233053 9105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:38.233141 9105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:38.257488 9105 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:36505
I20260811 18:51:38.258937 9105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:51:38.261164 9252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:36505 every 8 connection(s)
I20260811 18:51:38.263098 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 9105
I20260811 18:51:38.263248 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:51:38.263566 28111 external_mini_cluster.cc:1428] Setting key 3c4144a62b3744599d7732cec0963af2
Aug 11 18:51:38 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474298, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:51:38.285111 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.267868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:33385 (local address 127.27.115.254:44759)
0811 18:51:38.268040 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.268044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.269144 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 18:51:38.270497 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.270501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.270564 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.270619 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:38.272540 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.273400 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.275231 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.275601 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.275696 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.281072 (+ 5376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:38.281077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:38.281085 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:38.281116 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:38.282746 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:38.283436 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:38.283439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:38.283440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:38.283519 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:38.283904 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:38.283907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:38.283909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:38.284118 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:38.284231 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.284801 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.284944 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 18:51:38.286266 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.268392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44759 (local address 127.27.115.195:33385)
0811 18:51:38.268937 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.269052 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.270224 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.271044 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.271053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.271509 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:38.272354 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.272378 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.273580 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.273583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.275058 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.275071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.275942 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.275945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.278107 (+ 2162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.278975 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:38.278999 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:38.280815 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:38.282892 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:38.282901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:38.282922 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:38.283312 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:38.283615 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:38.283619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:38.283621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:38.283795 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:38.284231 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:38.284239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:38.284536 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.284751 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.285184 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":176,"threads_started":1}
I20260811 18:51:38.287840 9253 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44759
I20260811 18:51:38.288090 9253 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:38.288714 9253 heartbeater.cc:511] Master 127.27.115.254:44759 requested a full tablet report, sending...
I20260811 18:51:38.290900 8692 ts_manager.cc:194] Registered new tserver with Master: 7e27e27b9683465495da4ec55fab529d (127.27.115.195:36505)
I20260811 18:51:38.291646 8692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:33385
I20260811 18:51:38.302677 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:51:38.321261 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.310684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37032 (local address 127.27.115.254:44759)
0811 18:51:38.310835 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.310840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.311537 (+ 697us) server_negotiation.cc:407] Connection header received
0811 18:51:38.311774 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.311778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.311849 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.311906 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:38.313054 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.313892 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.315233 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.315565 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.315650 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.317774 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:38.317779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:38.317782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:38.317819 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:38.319510 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:38.319979 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:38.319984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:38.319986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:38.320074 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:38.320361 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:38.320365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:38.320367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:38.320570 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:38.320714 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.320966 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.321091 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:51:38.324815 8692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:51:38.329485 8692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:51:38.350991 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.340537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39653 (local address 127.0.0.1:36518)
0811 18:51:38.342736 (+ 2199us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.342796 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.342949 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.343822 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.343829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.343856 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:38.344015 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.344023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.346595 (+ 2572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.346599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.348000 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.348014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.350343 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.350348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.350495 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.350546 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.350649 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.350812 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1188,"spinlock_wait_cycles":6400,"thread_start_us":118,"threads_started":1}
I20260811 18:51:38.351655 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.340681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36518 (local address 127.27.115.193:39653)
0811 18:51:38.343270 (+ 2589us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.343278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.343303 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:51:38.343387 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.343393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.343628 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.344195 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:38.344369 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.346441 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.349211 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.350169 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.351252 (+ 1083us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.351383 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.351434 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.351518 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2465,"thread_start_us":111,"threads_started":1}
I20260811 18:51:38.351650 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.342331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.27.115.195:36505)
0811 18:51:38.343246 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.343258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.343577 (+ 319us) server_negotiation.cc:407] Connection header received
0811 18:51:38.343653 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.343658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.343836 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.343972 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:38.345216 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.346326 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.349281 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.350150 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.350278 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.350426 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.350811 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.350921 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":117,"threads_started":1}
I20260811 18:51:38.354703 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.341926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:36505 (local address 127.0.0.1:36014)
0811 18:51:38.343382 (+ 1456us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.343395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.343491 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.345008 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.345011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.345022 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:38.345099 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.345103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.346465 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.346469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.349099 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.349113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.350493 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.350497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.350610 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.350634 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.350706 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.354561 (+ 3855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":5092,"thread_start_us":78,"threads_started":1}
I20260811 18:51:38.355234 8858 tablet_service.cc:1467] Processing CreateTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 (DEFAULT_TABLE table=test-table [id=35897c76ae654b7c801f4f358f37eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:38.356482 8858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122a9e6e06cd434598e1a9a55fedb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:38.357797 9173 tablet_service.cc:1467] Processing CreateTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 (DEFAULT_TABLE table=test-table [id=35897c76ae654b7c801f4f358f37eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:38.358907 9173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122a9e6e06cd434598e1a9a55fedb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:38.365993 9275 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Bootstrap starting.
I20260811 18:51:38.367877 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.341565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45051 (local address 127.0.0.1:33730)
0811 18:51:38.342733 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.342799 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.342949 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.344805 (+ 1856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.344808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.344825 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:38.344907 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.344912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.352187 (+ 7275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.352193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.353340 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.353351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.367380 (+ 14029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.367384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.367516 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.367555 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.367617 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.367711 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"spinlock_wait_cycles":12928,"thread_start_us":101,"threads_started":1}
I20260811 18:51:38.368719 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.341724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.27.115.194:45051)
0811 18:51:38.344253 (+ 2529us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.344269 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.344292 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:51:38.344362 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.344366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.344529 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.344659 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:38.346798 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.347947 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.353545 (+ 5598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.354399 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.368239 (+ 13840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.368417 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.368479 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.368588 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2419,"thread_start_us":84,"threads_started":1}
I20260811 18:51:38.369076 9275 tablet_bootstrap.cc:654] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:38.370203 9276 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Bootstrap starting.
I20260811 18:51:38.372112 9011 tablet_service.cc:1467] Processing CreateTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 (DEFAULT_TABLE table=test-table [id=35897c76ae654b7c801f4f358f37eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:38.373152 9276 tablet_bootstrap.cc:654] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:38.373294 9011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122a9e6e06cd434598e1a9a55fedb395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:38.374019 9276 log.cc:824] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:38.378906 9275 log.cc:824] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:38.381633 9276 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: No bootstrap required, opened a new log
I20260811 18:51:38.381699 9279 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Bootstrap starting.
I20260811 18:51:38.381906 9276 ts_tablet_manager.cc:1397] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 18:51:38.384661 9279 tablet_bootstrap.cc:654] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:38.385653 9279 log.cc:824] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:38.388113 9279 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: No bootstrap required, opened a new log
I20260811 18:51:38.388204 9275 tablet_bootstrap.cc:492] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: No bootstrap required, opened a new log
I20260811 18:51:38.388361 9275 ts_tablet_manager.cc:1397] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.000s
I20260811 18:51:38.388362 9279 ts_tablet_manager.cc:1397] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:51:38.393206 9279 raft_consensus.cc:348] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.393154 9275 raft_consensus.cc:348] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.393509 9275 raft_consensus.cc:374] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:38.393576 9275 raft_consensus.cc:729] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118bb82d02f24867b8fd7f98d48d21fa, State: Initialized, Role: FOLLOWER
I20260811 18:51:38.393687 9279 raft_consensus.cc:374] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:38.393786 9279 raft_consensus.cc:729] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 674279fd4402452184a504a64a97baae, State: Initialized, Role: FOLLOWER
I20260811 18:51:38.394410 9279 consensus_queue.cc:260] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.394296 9276 raft_consensus.cc:348] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.394606 9276 raft_consensus.cc:374] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:38.394656 9276 raft_consensus.cc:729] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e27e27b9683465495da4ec55fab529d, State: Initialized, Role: FOLLOWER
I20260811 18:51:38.395136 9276 consensus_queue.cc:260] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.395937 9279 ts_tablet_manager.cc:1428] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 18:51:38.396068 9276 ts_tablet_manager.cc:1428] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260811 18:51:38.396483 9100 heartbeater.cc:503] Master 127.27.115.254:44759 was elected leader, sending a full tablet report...
I20260811 18:51:38.396495 9253 heartbeater.cc:503] Master 127.27.115.254:44759 was elected leader, sending a full tablet report...
I20260811 18:51:38.396708 9275 consensus_queue.cc:260] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.397852 9275 ts_tablet_manager.cc:1428] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:51:38.398334 8936 heartbeater.cc:503] Master 127.27.115.254:44759 was elected leader, sending a full tablet report...
W20260811 18:51:38.426031 8938 tablet.cc:2369] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:51:38 dist-test-slave-9jjr krb5kdc[8139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474297, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
W20260811 18:51:38.517169 9254 tablet.cc:2369] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:38.576982 9283 raft_consensus.cc:482] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:38.577247 9283 raft_consensus.cc:504] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.578840 9283 leader_election.cc:288] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 118bb82d02f24867b8fd7f98d48d21fa (127.27.115.193:39653), 674279fd4402452184a504a64a97baae (127.27.115.194:45051)
I20260811 18:51:38.584878 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.579355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39653 (local address 127.27.115.195:50735)
0811 18:51:38.579520 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.579539 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.579646 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.580852 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.580855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.580872 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:38.580997 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.581004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.582038 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.582041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.583361 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.583374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.584409 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.584429 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.584536 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.584561 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.584633 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.584713 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:51:38.585691 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.580440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:50735 (local address 127.27.115.193:39653)
0811 18:51:38.580598 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.580603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.580629 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:51:38.580690 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.580693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.580744 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.580840 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:38.581123 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.581892 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.583561 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.584149 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.584281 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.584337 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.585450 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.585546 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:51:38.586491 8876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122a9e6e06cd434598e1a9a55fedb395" candidate_uuid: "7e27e27b9683465495da4ec55fab529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118bb82d02f24867b8fd7f98d48d21fa" is_pre_election: true
I20260811 18:51:38.586872 8876 raft_consensus.cc:2454] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e27e27b9683465495da4ec55fab529d in term 0.
I20260811 18:51:38.587548 9125 leader_election.cc:302] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118bb82d02f24867b8fd7f98d48d21fa, 7e27e27b9683465495da4ec55fab529d; no voters:
I20260811 18:51:38.587947 9283 raft_consensus.cc:2781] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:51:38.588042 9283 raft_consensus.cc:482] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:51:38.588085 9283 raft_consensus.cc:3037] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:38.589619 9283 raft_consensus.cc:504] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
I20260811 18:51:38.590083 9283 leader_election.cc:288] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [CANDIDATE]: Term 1 election: Requested vote from peers 118bb82d02f24867b8fd7f98d48d21fa (127.27.115.193:39653), 674279fd4402452184a504a64a97baae (127.27.115.194:45051)
I20260811 18:51:38.591205 8876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122a9e6e06cd434598e1a9a55fedb395" candidate_uuid: "7e27e27b9683465495da4ec55fab529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118bb82d02f24867b8fd7f98d48d21fa"
I20260811 18:51:38.591379 8876 raft_consensus.cc:3037] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:38.592798 8876 raft_consensus.cc:2454] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e27e27b9683465495da4ec55fab529d in term 1.
I20260811 18:51:38.593350 9125 leader_election.cc:302] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118bb82d02f24867b8fd7f98d48d21fa, 7e27e27b9683465495da4ec55fab529d; no voters:
I20260811 18:51:38.593693 9283 raft_consensus.cc:2781] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:38.593979 9283 raft_consensus.cc:686] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 1 LEADER]: Becoming Leader. State: Replica: 7e27e27b9683465495da4ec55fab529d, State: Running, Role: LEADER
I20260811 18:51:38.594436 9283 consensus_queue.cc:237] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } }
W20260811 18:51:38.594765 9101 tablet.cc:2369] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:38.598280 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.580631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:55281 (local address 127.27.115.194:45051)
0811 18:51:38.580794 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.580799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.580814 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:51:38.580868 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.580871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.580922 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.580997 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:38.594465 (+ 13468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.595249 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.596691 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.597278 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.597520 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.597587 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.598013 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.598111 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:51:38.598785 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.579763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45051 (local address 127.27.115.195:55281)
0811 18:51:38.580445 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.580462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.580566 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.594024 (+ 13458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:38.594029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:38.594066 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:38.594248 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.594255 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.595421 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.595424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.596529 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:38.596542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.597740 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.597744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:38.597850 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.597876 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:51:38.598526 (+ 650us) client_negotiation.cc:240] Negotiation successful
0811 18:51:38.598620 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":126,"threads_started":1}
I20260811 18:51:38.598457 8692 catalog_manager.cc:5495] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e27e27b9683465495da4ec55fab529d (127.27.115.195). New cstate: current_term: 1 leader_uuid: "7e27e27b9683465495da4ec55fab529d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e27e27b9683465495da4ec55fab529d" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 36505 } health_report { overall_health: HEALTHY } } }
I20260811 18:51:38.599509 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122a9e6e06cd434598e1a9a55fedb395" candidate_uuid: "7e27e27b9683465495da4ec55fab529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674279fd4402452184a504a64a97baae" is_pre_election: true
I20260811 18:51:38.599840 9038 raft_consensus.cc:2454] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e27e27b9683465495da4ec55fab529d in term 0.
I20260811 18:51:38.600225 9045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122a9e6e06cd434598e1a9a55fedb395" candidate_uuid: "7e27e27b9683465495da4ec55fab529d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674279fd4402452184a504a64a97baae"
I20260811 18:51:38.600338 9045 raft_consensus.cc:3037] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:38.601858 9045 raft_consensus.cc:2454] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e27e27b9683465495da4ec55fab529d in term 1.
I20260811 18:51:38.652865 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.646948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.27.115.195:36505)
0811 18:51:38.647168 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:38.647174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:38.647191 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:51:38.647317 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:38.647320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:38.647398 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:38.647526 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:51:38.647997 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.648865 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.650293 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:38.650672 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:38.650778 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:38.651254 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:51:38.651911 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:51:38.652046 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:38.652621 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 18:51:38.652702 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 18:51:38.671627 9045 raft_consensus.cc:1261] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 1 FOLLOWER]: Refusing update from remote peer 7e27e27b9683465495da4ec55fab529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:38.672683 9283 consensus_queue.cc:1035] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "674279fd4402452184a504a64a97baae" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:38.681785 8876 raft_consensus.cc:1261] T 122a9e6e06cd434598e1a9a55fedb395 P 118bb82d02f24867b8fd7f98d48d21fa [term 1 FOLLOWER]: Refusing update from remote peer 7e27e27b9683465495da4ec55fab529d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:38.684408 9291 mvcc.cc:205] Tried to move back new op lower bound from 7317398727347191808 to 7317398727046676480. Current Snapshot: MvccSnapshot[applied={T|T < 7317398727347191808}]
I20260811 18:51:38.690534 9283 consensus_queue.cc:1035] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [LEADER]: Connected to new peer: Peer: permanent_uuid: "118bb82d02f24867b8fd7f98d48d21fa" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:38.699062 8692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:38.699337 8692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:38.701457 9292 mvcc.cc:205] Tried to move back new op lower bound from 7317398727347191808 to 7317398727046676480. Current Snapshot: MvccSnapshot[applied={T|T < 7317398727347191808 or (T in {7317398727347191808})}]
I20260811 18:51:38.702903 8692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d11ce1cd11dc497ea4c0dd66e32a4ae7: Sending DeleteTablet for 3 replicas of tablet 122a9e6e06cd434598e1a9a55fedb395
I20260811 18:51:38.703948 9011 tablet_service.cc:1514] Processing DeleteTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:38 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:33730
I20260811 18:51:38.705889 8858 tablet_service.cc:1514] Processing DeleteTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:38 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36518
I20260811 18:51:38.706023 9300 tablet_replica.cc:330] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: stopping tablet replica
I20260811 18:51:38.706470 9300 raft_consensus.cc:2229] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:38.706888 9300 raft_consensus.cc:2258] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:38.706904 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 8780
I20260811 18:51:38.708758 9300 ts_tablet_manager.cc:1905] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:38.712216 9300 ts_tablet_manager.cc:1918] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:38.712355 9300 log.cc:1196] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/122a9e6e06cd434598e1a9a55fedb395
I20260811 18:51:38.712841 9300 ts_tablet_manager.cc:1939] T 122a9e6e06cd434598e1a9a55fedb395 P 674279fd4402452184a504a64a97baae: Deleting consensus metadata
I20260811 18:51:38.714426 8676 catalog_manager.cc:4841] TS 674279fd4402452184a504a64a97baae (127.27.115.194:45051): tablet 122a9e6e06cd434598e1a9a55fedb395 (table test-table [id=35897c76ae654b7c801f4f358f37eaa4]) successfully deleted
I20260811 18:51:38.718715 9173 tablet_service.cc:1514] Processing DeleteTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:38 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36014
I20260811 18:51:38.719241 9170 tablet_service.cc:1514] Processing DeleteTablet for tablet 122a9e6e06cd434598e1a9a55fedb395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:38 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36014
W20260811 18:51:38.727489 9125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:39653 (error 108)
I20260811 18:51:38.728578 9302 tablet_replica.cc:330] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d: stopping tablet replica
I20260811 18:51:38.729106 9302 raft_consensus.cc:2229] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:51:38.729588 8677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:39653 (error 108)
W20260811 18:51:38.731380 8678 connection.cc:535] server connection from 127.27.115.193:43163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:51:38.732453 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:51:38.730710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39653 (local address 127.27.115.195:60413)
0811 18:51:38.731004 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.731022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.731139 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.732309 (+ 1170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39653: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":176}
W20260811 18:51:38.733314 9125 consensus_peers.cc:487] T 122a9e6e06cd434598e1a9a55fedb395 P 7e27e27b9683465495da4ec55fab529d -> Peer 118bb82d02f24867b8fd7f98d48d21fa (127.27.115.193:39653): Couldn't send request to peer 118bb82d02f24867b8fd7f98d48d21fa. Status: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39653: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:51:38.734754 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 8943
I20260811 18:51:38.759338 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 9105
I20260811 18:51:38.762502 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:38.762034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:39653 (local address 127.0.0.1:36522)
0811 18:51:38.762295 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.762376 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260811 18:51:38.762872 8677 catalog_manager.cc:4584] TS 118bb82d02f24867b8fd7f98d48d21fa (127.27.115.193:39653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 122a9e6e06cd434598e1a9a55fedb395: Network error: Client connection negotiation failed: client connection to 127.27.115.193:39653: connect: Connection refused (error 111)
W20260811 18:51:38.770751 9272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:51:38.770044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:36505 (local address 127.0.0.1:36028)
0811 18:51:38.770295 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:38.770311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:38.770414 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:38.770622 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.195:36505: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 18:51:38.771517 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 8657
I20260811 18:51:38.781800 28111 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:51:39.128522 28111 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:51:39.128814 28111 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:51:39.475358 28111 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:51:39.475 UTC [8172] LOG: received smart shutdown request
2026-08-11 18:51:39.479 UTC [8172] LOG: background worker "logical replication launcher" (PID 8183) exited with exit code 1
2026-08-11 18:51:39.480 UTC [8176] LOG: shutting down
2026-08-11 18:51:39.668 UTC [8172] LOG: database system is shut down
2026-08-11T18:51:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56747 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:51:39 dist-test-slave-9jjr krb5kdc[9314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:51:39 dist-test-slave-9jjr krb5kdc[9314](info): set up 2 sockets
Aug 11 18:51:39 dist-test-slave-9jjr krb5kdc[9314](info): commencing operation
krb5kdc: starting...
W20260811 18:51:40.961879 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:51:41 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:51:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangeradmin_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangeradmin_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangerlookup_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangerlookup_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
I20260811 18:51:41.169160 28111 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4WGUPP/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/postgres -l logfile start
2026-08-11 18:51:44.246 UTC [9348] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:51:44.246 UTC [9348] LOG: listening on IPv4 address "127.27.115.212", port 38053
2026-08-11 18:51:44.251 UTC [9348] LOG: listening on Unix socket "/tmp/.s.PGSQL.38053"
2026-08-11 18:51:44.283 UTC [9351] LOG: database system was shut down at 2026-08-11 18:51:42 UTC
2026-08-11 18:51:44.288 UTC [9348] LOG: database system is ready to accept connections
I20260811 18:51:45.329416 28111 mini_postgres.cc:96] Postgres bound to 38053
2026-08-11 18:51:45.337 UTC [9359] FATAL: database "slave" does not exist
127.27.115.212:38053 - accepting connections
I20260811 18:51:45.338370 28111 mini_ranger.cc:161] Starting Ranger...
I20260811 18:51:45.358777 28111 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:51:46.025429 28111 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:51:46.025544 28111 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin
2026-08-11 18:51:46,356 [I] DB FLAVOR :POSTGRES
2026-08-11 18:51:46,356 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:51:46,357 [I] Checking connection..
2026-08-11 18:51:46,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:51:46,656 [I] Checking connection passed.
2026-08-11 18:51:46,657 [I] --------- Verifying version history table ---------
2026-08-11 18:51:46,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:51:46,965 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:51:46,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:51:47,266 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:51:47,267 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:51:47,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:51:47.529 UTC [9438] WARNING: there is no transaction in progress
2026-08-11 18:51:47,539 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:51:47,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:51:47,807 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:51:47,807 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:51:47,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:51:48,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:51:48,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9jjr.c.gcp-upstream.internal','N') ;"
2026-08-11 18:51:48,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:51:48,934 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:51:48,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:51:49,213 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:51:49,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:51:49,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:51:49,741 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:51:49,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:51:51.028 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.052 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.076 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.098 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.121 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.146 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.220 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.229 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.247 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.260 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.274 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.291 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.302 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.311 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.322 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:51.334 UTC [9557] WARNING: there is no transaction in progress
2026-08-11 18:51:52,279 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:51:52,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:51:52,533 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:51:52,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:51:52,825 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:51:52,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:51:53,102 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:51:53,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:51:53,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:51:53,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9jjr.c.gcp-upstream.internal';"
2026-08-11 18:51:53,919 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:51:53,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:51:54,193 [I] DB_PATCHES have already been applied
I20260811 18:51:54.200263 28111 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4WGUPP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:51:54.200387 28111 mini_ranger.cc:193] Using host: 127.27.115.212
I20260811 18:51:54.205329 28111 mini_ranger.cc:227] Ranger admin URL: http://127.27.115.212:52813
Aug 11, 2026 6:51:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:51:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:51:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:51:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:51:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:51:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:51:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:51:55.475070 28111 mini_ranger.cc:160] Time spent starting Ranger: real 10.137s user 0.007s sys 0.007s
Aug 11, 2026 6:52:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:52:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:52:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:52:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:52:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:52:19.785145 28111 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangerkms_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/rangerkms_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.115.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.115.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.115.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/HTTP_127.27.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:52:19 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474339, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:52:19.893353 28111 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:52:19.905701 28111 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:52:21.076370 28111 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:52:21.076470 28111 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms
I20260811 18:52:21.076493 28111 mini_ranger_kms.cc:227] Using postgres at 127.27.115.212:38053
2026-08-11 18:52:21,278 [I] DB FLAVOR :POSTGRES
2026-08-11 18:52:21,279 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:52:21,279 [I] Checking connection
2026-08-11 18:52:21,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:52:21,545 [I] connection success
2026-08-11 18:52:21,545 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:52:21,545 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:52:21,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:52:21,816 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:52:21,817 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:52:21,817 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:52:21,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.115.212:38053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:52:22,100 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:52:22.687134 28111 mini_ranger_kms.cc:310] Created kms service
I20260811 18:52:22.924548 28111 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:52:22.983209 28111 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:52:22 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474339, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:23.423691 28111 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:52:23.423863 28111 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms:/tmp/dist-test-task4WGUPP/build/debug/bin/postgresql.jar:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4WGUPP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:52:23.423885 28111 mini_ranger_kms.cc:240] Using host: 127.27.115.212
I20260811 18:52:23.429430 28111 mini_ranger_kms.cc:276] Ranger KMS PID: 9762
I20260811 18:52:23.429555 28111 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.115.212:34443
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:52:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:52:25 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:25.900365 28111 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.007s user 0.004s sys 0.182s
I20260811 18:52:25.900507 28111 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:52:25.900583 28111 mini_ranger_kms.cc:401] 127.27.115.212:34443/kms/v1/keys
Aug 11 18:52:26 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
Aug 11 18:52:26 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.115.212@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
Aug 11 18:52:29 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:52:29.094592 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44371
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:44371
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:29.217466 9819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:29.217792 9819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:29.217836 9819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:29.222378 9819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:29.222460 9819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:29.222481 9819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:29.222497 9819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:29.222512 9819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:29.228581 9819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.115.212:34443/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:44371
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:29.229840 9819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:29.231071 9819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:29.241158 9825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:29.241159 9824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:29.241391 9819 server_base.cc:1034] running on GCE node
W20260811 18:52:29.241568 9827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:29.242365 9819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:29.243885 9819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:29.245093 9819 hybrid_clock.cc:648] HybridClock initialized: now 1786474349245069 us; error 40 us; skew 500 ppm
Aug 11 18:52:29 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:29.260018 9819 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:29.261276 9819 webserver.cc:460] Webserver started at http://127.27.115.254:38123/ using document root <none> and password file <none>
I20260811 18:52:29.261917 9819 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:29.262002 9819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:29.262286 9819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:52:29 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:29.313345 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "ecab900f9df641ed878c1d00477f2947"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-9jjr"
server_key: "ec683877fdef76f31c2e6282624ec1ba"
server_key_iv: "04a28f2a912ce1226618c3885143b9b1"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:29.314014 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "ecab900f9df641ed878c1d00477f2947"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-9jjr"
server_key: "ec683877fdef76f31c2e6282624ec1ba"
server_key_iv: "04a28f2a912ce1226618c3885143b9b1"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:29.318282 9819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:52:29.334231 9834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:29.335750 9819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:52:29.335920 9819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "ecab900f9df641ed878c1d00477f2947"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-9jjr"
server_key: "ec683877fdef76f31c2e6282624ec1ba"
server_key_iv: "04a28f2a912ce1226618c3885143b9b1"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:29.336081 9819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:29.350883 9819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:29.354283 9819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:29.354503 9819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:29.363240 9819 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:44371
I20260811 18:52:29.363255 9895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:44371 every 8 connection(s)
I20260811 18:52:29.364387 9819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:29.367647 9896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:29.373811 9896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Bootstrap starting.
I20260811 18:52:29.374176 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 9819
I20260811 18:52:29.374317 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:52:29.374650 28111 external_mini_cluster.cc:1428] Setting key c642125dd7c55cd9360448a84864eb90
I20260811 18:52:29.377032 9896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:29.377976 9896 log.cc:824] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:29.380672 9896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: No bootstrap required, opened a new log
Aug 11 18:52:29 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:29.385910 9896 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:29.386400 9896 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:29.386521 9896 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecab900f9df641ed878c1d00477f2947, State: Initialized, Role: FOLLOWER
I20260811 18:52:29.387156 9896 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:29.387408 9896 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:29.387497 9896 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:29.387621 9896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:29.389061 9896 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:29.389549 9896 leader_election.cc:302] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecab900f9df641ed878c1d00477f2947; no voters:
I20260811 18:52:29.389982 9896 leader_election.cc:288] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:29.390190 9902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:29.390479 9902 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 LEADER]: Becoming Leader. State: Replica: ecab900f9df641ed878c1d00477f2947, State: Running, Role: LEADER
I20260811 18:52:29.390854 9902 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:29.391291 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:29.376335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.27.115.254:44371)
0811 18:52:29.376905 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:29.376927 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:29.376974 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:52:29.377915 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:29.377939 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:29.378420 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:29.378807 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:29.379686 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:29.380664 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:29.381586 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:29.382077 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:29.382317 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:29.385059 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:29.385066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:29.385085 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:29.385124 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:29.387199 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:29.387973 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:29.387985 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:29.387991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:29.388121 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:29.388442 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:29.388448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:29.388450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:29.388922 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:29.389141 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:29.389816 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 18:52:29.390179 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":171,"threads_started":1}
I20260811 18:52:29.391496 9896 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:29.392223 9902 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecab900f9df641ed878c1d00477f2947. Latest consensus state: current_term: 1 leader_uuid: "ecab900f9df641ed878c1d00477f2947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } } }
I20260811 18:52:29.392444 9902 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:29.392894 9906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:29.394160 9903 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecab900f9df641ed878c1d00477f2947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } } }
I20260811 18:52:29.394274 9903 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:29.397022 9906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:29.397571 9896 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:52:29.410859 9906 catalog_manager.cc:1347] Generated new cluster ID: c4c47cdb1e174377b5f0608f3b25a461
I20260811 18:52:29.410962 9906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:29.433066 9906 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:29.434906 9906 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:29.447067 9906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Generated new TSK 0
I20260811 18:52:29.448233 9906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:52:30.269767 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:30.453399 9947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:30.453825 9947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:30.453891 9947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:30.460341 9947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:30.460443 9947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:30.460536 9947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:52:30.469336 9947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:30.471098 9947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:30.472647 9947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:30.487824 9959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:30.487944 9954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:30.487825 9956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:30.488251 9947 server_base.cc:1034] running on GCE node
I20260811 18:52:30.488915 9947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:30.489727 9947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:30.490959 9947 hybrid_clock.cc:648] HybridClock initialized: now 1786474350490904 us; error 76 us; skew 500 ppm
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:30.495376 9947 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:52:30.496656 9947 webserver.cc:460] Webserver started at http://127.27.115.193:38009/ using document root <none> and password file <none>
I20260811 18:52:30.497428 9947 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:30.497520 9947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:30.497756 9947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:30.523010 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "c9c4020d20fa4abc914056c7605de10b"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "4641757c8592dbc9f37d54685f5cff51"
server_key_iv: "b22c56758cb15fa6507749acac92cb77"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:30.524004 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "c9c4020d20fa4abc914056c7605de10b"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "4641757c8592dbc9f37d54685f5cff51"
server_key_iv: "b22c56758cb15fa6507749acac92cb77"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:30.529687 9947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:52:30.574612 9966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:30.577010 9947 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.000s
I20260811 18:52:30.577217 9947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "c9c4020d20fa4abc914056c7605de10b"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "4641757c8592dbc9f37d54685f5cff51"
server_key_iv: "b22c56758cb15fa6507749acac92cb77"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:30.577402 9947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:30.595837 9947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:30.599335 9947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:30.599550 9947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:30.600209 9947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:30.601176 9947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:30.601266 9947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:30.601361 9947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:30.601413 9947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:30.613112 9947 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:33491
I20260811 18:52:30.614590 9947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:30.614903 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 9947
I20260811 18:52:30.615041 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:30.615409 28111 external_mini_cluster.cc:1428] Setting key 6c6b5f56afb8f1e3d9577e427576d57b
I20260811 18:52:30.616272 10102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:33491 every 8 connection(s)
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:30.641825 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:30.619748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.193:49183)
0811 18:52:30.620999 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:30.621059 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:30.622109 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:30.624712 (+ 2603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:30.624722 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:30.625121 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:30.625839 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:30.625858 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:30.627130 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:30.627133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:30.628586 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:30.628598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:30.629472 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:30.629477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:30.629707 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:30.633280 (+ 3573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:30.633306 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:30.635293 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:30.637910 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:30.637919 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:30.637942 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:30.638396 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:30.638774 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:30.638777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:30.638779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:30.638991 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:30.639553 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:30.639560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:30.639963 (+ 403us) client_negotiation.cc:769] Sending connection context
0811 18:52:30.640204 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 18:52:30.640602 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":146,"threads_started":1}
I20260811 18:52:30.642758 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:30.619617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:49183 (local address 127.27.115.254:44371)
0811 18:52:30.624110 (+ 4493us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:30.624115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:30.624134 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:52:30.624187 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:30.624191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:30.624248 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:30.624344 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:30.626042 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:30.626950 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:30.628764 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:30.629108 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:30.629254 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:30.635594 (+ 6340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:30.635598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:30.635601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:30.635654 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:30.637716 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:30.638538 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:30.638542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:30.638544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:30.638661 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:30.639117 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:30.639120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:30.639122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:30.639378 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:30.639492 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:30.642406 (+ 2914us) server_negotiation.cc:299] Negotiation successful
0811 18:52:30.642571 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4395,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:52:30.649652 10103 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:30.649991 10103 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:30.650934 10103 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:30.653373 9856 ts_manager.cc:194] Registered new tserver with Master: c9c4020d20fa4abc914056c7605de10b (127.27.115.193:33491)
I20260811 18:52:30.655165 9856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:49183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:52:30.692169 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:52:30.895814 10110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:30.896425 10110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:30.896565 10110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:30.903685 10110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:30.903975 10110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:30.904130 10110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:52:30.912730 10110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:30.914311 10110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:30.915777 10110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:30.929697 10120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:30.929924 10118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:30.930356 10117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:30.931533 10110 server_base.cc:1034] running on GCE node
I20260811 18:52:30.932140 10110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:30.933110 10110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:30.934311 10110 hybrid_clock.cc:648] HybridClock initialized: now 1786474350934288 us; error 41 us; skew 500 ppm
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:30.944043 10110 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:52:30.945588 10110 webserver.cc:460] Webserver started at http://127.27.115.194:45337/ using document root <none> and password file <none>
I20260811 18:52:30.946770 10110 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:30.946995 10110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:30.947371 10110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:52:30 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:30.970208 10110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "e65c1752f6094da4b23091cafd6bbbe4"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "8b6d3350b4251e9be383f167f0284a6f"
server_key_iv: "24c531d3e57f9fdf7f48b5aafe4522ed"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:30.972594 10110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "e65c1752f6094da4b23091cafd6bbbe4"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "8b6d3350b4251e9be383f167f0284a6f"
server_key_iv: "24c531d3e57f9fdf7f48b5aafe4522ed"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:30.979152 10110 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 18:52:31.009943 10127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.012322 10110 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 18:52:31.012540 10110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "e65c1752f6094da4b23091cafd6bbbe4"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "8b6d3350b4251e9be383f167f0284a6f"
server_key_iv: "24c531d3e57f9fdf7f48b5aafe4522ed"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:31.012755 10110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:31.047719 10110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:31.052392 10110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:31.052706 10110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:31.053726 10110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:31.055210 10110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:31.055342 10110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 18:52:31.055461 10110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:31.055505 10110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.067790 10110 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:33499
I20260811 18:52:31.067965 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:33499 every 8 connection(s)
I20260811 18:52:31.068933 10110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:31.077987 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10110
I20260811 18:52:31.078375 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:31.078719 28111 external_mini_cluster.cc:1428] Setting key a147197a9e0f34b1c9a9db4dda026045
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:31.085078 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.071137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:46139 (local address 127.27.115.254:44371)
0811 18:52:31.071315 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:31.071320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:31.072210 (+ 890us) server_negotiation.cc:407] Connection header received
0811 18:52:31.073225 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:31.073228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:31.073290 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:31.073404 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:31.074846 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.075395 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.076626 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.076970 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.077054 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.080278 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:31.080282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:31.080285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:31.080314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:31.082282 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.082982 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.082986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.082988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.083104 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.083652 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.083656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.083658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.083900 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:31.083977 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:31.084756 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 18:52:31.084903 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:52:31.086867 10263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.071553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.194:46139)
0811 18:52:31.072048 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:31.072103 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:31.072999 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:31.073598 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:31.073604 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:31.073897 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:31.074672 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:31.074697 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.075536 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.075539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:31.076439 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:31.076449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.077285 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.077290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:31.077497 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.078301 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:31.078323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:31.080068 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:31.082427 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:31.082434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:31.082454 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:31.082852 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:31.083241 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:31.083244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:31.083246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:31.083443 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:31.084066 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:31.084073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:31.084418 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 18:52:31.085355 (+ 937us) client_negotiation.cc:240] Negotiation successful
0811 18:52:31.085741 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":141,"threads_started":1}
I20260811 18:52:31.088461 10260 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:31.088732 10260 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:31.089346 10260 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:31.090745 9856 ts_manager.cc:194] Registered new tserver with Master: e65c1752f6094da4b23091cafd6bbbe4 (127.27.115.194:33499)
I20260811 18:52:31.091594 9856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:46139
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:52:31.121726 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474350, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
W20260811 18:52:31.301708 10266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:31.302564 10266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:31.302800 10266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:31.311408 10266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:31.311597 10266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:31.311753 10266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:52:31.320669 10266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:31.322551 10266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:31.324393 10266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:31.340260 10273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:31.340605 10266 server_base.cc:1034] running on GCE node
W20260811 18:52:31.340376 10274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:31.341709 10276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:31.342415 10266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:31.343343 10266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:31.346621 10266 hybrid_clock.cc:648] HybridClock initialized: now 1786474351346530 us; error 93 us; skew 500 ppm
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:31.350716 10266 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:52:31.352151 10266 webserver.cc:460] Webserver started at http://127.27.115.195:46581/ using document root <none> and password file <none>
I20260811 18:52:31.353019 10266 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:31.353193 10266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:31.353499 10266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:31.390807 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "1d55ea9b96aa41899b4f75bc3e1f1830"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-9jjr"
server_key: "2453ed35ddad9a920837c461fa931a60"
server_key_iv: "ad0e2db8be8fb3e69fa0fee6d2e52034"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:31.391885 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "1d55ea9b96aa41899b4f75bc3e1f1830"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-9jjr"
server_key: "2453ed35ddad9a920837c461fa931a60"
server_key_iv: "ad0e2db8be8fb3e69fa0fee6d2e52034"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:31.399979 10266 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.004s
I20260811 18:52:31.434435 10284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.437254 10266 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260811 18:52:31.437489 10266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "1d55ea9b96aa41899b4f75bc3e1f1830"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-9jjr"
server_key: "2453ed35ddad9a920837c461fa931a60"
server_key_iv: "ad0e2db8be8fb3e69fa0fee6d2e52034"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:31.437690 10266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:31.472177 10266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:31.476073 10266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:31.476328 10266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:31.477063 10266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:31.478169 10266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:31.478240 10266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.478304 10266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:31.478334 10266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.490628 10266 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:39157
I20260811 18:52:31.492146 10266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:31.494984 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10266
I20260811 18:52:31.495118 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:31.495437 28111 external_mini_cluster.cc:1428] Setting key 0e79c71ff787b0b8221dee4bd0b9304a
I20260811 18:52:31.496059 10409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:39157 every 8 connection(s)
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:31.513752 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.496757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:35011 (local address 127.27.115.254:44371)
0811 18:52:31.496909 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:31.496914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:31.497931 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 18:52:31.499557 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:31.499561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:31.499626 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:31.499679 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:31.501696 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.502593 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.504638 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.504969 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.505092 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.508687 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:31.508692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:31.508695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:31.508733 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:31.510743 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.511509 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.511512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.511514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.511624 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.512212 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.512215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.512217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.512499 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:31.512617 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:31.513367 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 18:52:31.513551 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:52:31.515131 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.497176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.195:35011)
0811 18:52:31.497761 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:31.497823 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:31.499268 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:31.500232 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:31.500242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:31.500664 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:31.501463 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:31.501485 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.502789 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.502793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:31.504436 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:31.504453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.505188 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.505194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:31.505440 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.506345 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:31.506370 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:31.508407 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:31.510918 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:31.510925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:31.510948 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:31.511392 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:31.511755 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:31.511758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:31.511760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:31.512082 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:31.512650 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:31.512661 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:31.513043 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 18:52:31.513286 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 18:52:31.513711 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":134,"threads_started":1}
I20260811 18:52:31.516922 10410 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:31.517263 10410 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:31.518023 10410 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:31.519589 9856 ts_manager.cc:194] Registered new tserver with Master: 1d55ea9b96aa41899b4f75bc3e1f1830 (127.27.115.195:39157)
I20260811 18:52:31.520434 9856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:35011
I20260811 18:52:31.523835 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:31.524631 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 9947
I20260811 18:52:31.535403 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10110
I20260811 18:52:31.548235 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10266
I20260811 18:52:31.558796 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 9819
I20260811 18:52:31.567585 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44371
--webserver_interface=127.27.115.254
--webserver_port=38123
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:44371
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:31.691749 10414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:31.692058 10414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:31.692101 10414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:31.696318 10414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:31.696395 10414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:31.696415 10414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:31.696432 10414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:31.696447 10414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:31.702339 10414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:44371
--ranger_config_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:31.703581 10414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:31.704816 10414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:31.714128 10421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:31.714169 10424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:31.714176 10419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:31.714450 10414 server_base.cc:1034] running on GCE node
I20260811 18:52:31.715279 10414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:31.716549 10414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:31.717796 10414 hybrid_clock.cc:648] HybridClock initialized: now 1786474351717746 us; error 69 us; skew 500 ppm
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:31.721212 10414 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:31.722432 10414 webserver.cc:460] Webserver started at http://127.27.115.254:38123/ using document root <none> and password file <none>
I20260811 18:52:31.723274 10414 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:31.723407 10414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:52:31 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:31.808159 10414 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:31.810708 10432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.812165 10414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:31.812288 10414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "ecab900f9df641ed878c1d00477f2947"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-9jjr"
server_key: "ec683877fdef76f31c2e6282624ec1ba"
server_key_iv: "04a28f2a912ce1226618c3885143b9b1"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:31.812923 10414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:31.831171 10414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:31.834651 10414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:31.834880 10414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:31.843672 10414 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:44371
I20260811 18:52:31.843703 10490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:44371 every 8 connection(s)
I20260811 18:52:31.845360 10414 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:31.847894 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10414
I20260811 18:52:31.848381 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:33491
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:44371
--webserver_port=38009
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:31.850536 10491 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:52:31.852412 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Bootstrap starting.
I20260811 18:52:31.870937 10491 log.cc:824] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:31.879468 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:52:31.880064 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Bootstrap complete.
I20260811 18:52:31.884649 10491 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:31.885115 10491 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecab900f9df641ed878c1d00477f2947, State: Initialized, Role: FOLLOWER
I20260811 18:52:31.885730 10491 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:31.885885 10491 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:31.885974 10491 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:31.886101 10491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:52:31.888242 10491 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:31.888604 10491 leader_election.cc:302] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecab900f9df641ed878c1d00477f2947; no voters:
I20260811 18:52:31.888950 10491 leader_election.cc:288] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:52:31.889120 10496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:52:31.889358 10496 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [term 2 LEADER]: Becoming Leader. State: Replica: ecab900f9df641ed878c1d00477f2947, State: Running, Role: LEADER
I20260811 18:52:31.889693 10496 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } }
I20260811 18:52:31.890389 10491 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:31.891206 10497 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ecab900f9df641ed878c1d00477f2947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } } }
I20260811 18:52:31.891335 10497 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:31.891654 10498 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecab900f9df641ed878c1d00477f2947. Latest consensus state: current_term: 2 leader_uuid: "ecab900f9df641ed878c1d00477f2947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecab900f9df641ed878c1d00477f2947" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44371 } } }
I20260811 18:52:31.891736 10498 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:31.892102 10504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:31.895478 10504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:31.896152 10504 catalog_manager.cc:1259] Loaded cluster ID: c4c47cdb1e174377b5f0608f3b25a461
I20260811 18:52:31.896291 10504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:31.896175 10491 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:52:31.901685 10504 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:31.903527 10504 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Loaded TSK: 0
I20260811 18:52:31.906230 10504 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:52:31.983083 10493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:31.983424 10493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:31.983505 10493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:31.989915 10493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:31.990006 10493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:31.990101 10493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:52:31.999595 10493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:33491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=38009
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:32.001196 10493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:32.002720 10493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:32.016419 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.016422 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.017330 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:32.017688 10493 server_base.cc:1034] running on GCE node
I20260811 18:52:32.018321 10493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:32.019280 10493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:32.020478 10493 hybrid_clock.cc:648] HybridClock initialized: now 1786474352020456 us; error 43 us; skew 500 ppm
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:32.024734 10493 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:52:32.026305 10493 webserver.cc:460] Webserver started at http://127.27.115.193:38009/ using document root <none> and password file <none>
I20260811 18:52:32.027123 10493 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:32.027261 10493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:32.053584 10493 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:52:32.058830 10543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.060638 10493 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:52:32.060855 10493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "c9c4020d20fa4abc914056c7605de10b"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "4641757c8592dbc9f37d54685f5cff51"
server_key_iv: "b22c56758cb15fa6507749acac92cb77"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:32.061708 10493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:32.085065 10493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:32.090085 10493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:32.090358 10493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:32.091189 10493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:32.092267 10493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:32.092331 10493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.092386 10493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:32.092408 10493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.104141 10493 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:33491
I20260811 18:52:32.104159 10673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:33491 every 8 connection(s)
I20260811 18:52:32.105342 10493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:32.109261 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10493
I20260811 18:52:32.109733 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:33499
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:44371
--webserver_port=45337
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:32.138916 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.107945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.193:36695)
0811 18:52:32.112142 (+ 4197us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:32.112225 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:32.113427 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:32.121225 (+ 7798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:32.121236 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:32.121657 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:32.122499 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.122525 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.125063 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.125070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.126735 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.126751 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.127914 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.127921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.128145 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.128926 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:32.128949 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:32.130891 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:32.134150 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.134158 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.134175 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.134590 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.135083 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.135087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.135089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.135293 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.136352 (+ 1059us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:32.136371 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:32.136738 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 18:52:32.136956 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 18:52:32.137394 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3908,"thread_start_us":134,"threads_started":1}
I20260811 18:52:32.141067 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.107771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:36695 (local address 127.27.115.254:44371)
0811 18:52:32.108445 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.108468 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.119551 (+ 11083us) server_negotiation.cc:407] Connection header received
0811 18:52:32.119999 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.120021 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.120582 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.122822 (+ 2240us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.123233 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.124830 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.126922 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.127697 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.131197 (+ 3500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.131492 (+ 295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.131500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.131520 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:32.131556 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.133952 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.134777 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.134787 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.134792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.134956 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.135425 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.135430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.135434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.136070 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.139458 (+ 3388us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.139660 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.140026 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":141,"threads_started":1}
I20260811 18:52:32.144076 10674 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:32.144387 10674 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.145036 10674 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.147732 10674 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:44371 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:52:32.147886 10674 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.148118 10674 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:32.149156 10674 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.149386 10674 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.150324 10674 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:52:32.270344 10678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:32.270659 10678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:32.270709 10678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:32.275418 10678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:32.275507 10678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:32.275607 10678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:52:32.281996 10678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:33499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=45337
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:32.283381 10678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:32.284754 10678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:32.297405 10687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.297385 10685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:32.297773 10678 server_base.cc:1034] running on GCE node
W20260811 18:52:32.297463 10684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:32.298528 10678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:32.299384 10678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:32.300591 10678 hybrid_clock.cc:648] HybridClock initialized: now 1786474352300552 us; error 59 us; skew 500 ppm
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:32.304039 10678 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:52:32.305326 10678 webserver.cc:460] Webserver started at http://127.27.115.194:45337/ using document root <none> and password file <none>
I20260811 18:52:32.306159 10678 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:32.306279 10678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:32.330554 10678 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:52:32.333936 10696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.335871 10678 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:52:32.336081 10678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "e65c1752f6094da4b23091cafd6bbbe4"
format_stamp: "Formatted at 2026-08-11 18:52:30 on dist-test-slave-9jjr"
server_key: "8b6d3350b4251e9be383f167f0284a6f"
server_key_iv: "24c531d3e57f9fdf7f48b5aafe4522ed"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:32.337126 10678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:32.349335 10678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:32.353073 10678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:32.353322 10678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:32.354054 10678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:32.355161 10678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:32.355224 10678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.355283 10678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:32.355307 10678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.368364 10678 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:33499
I20260811 18:52:32.368541 10832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:33499 every 8 connection(s)
I20260811 18:52:32.369822 10678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:32.377599 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10678
I20260811 18:52:32.378274 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.115.212:34443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:39157
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:44371
--webserver_port=46581
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:32.389869 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.374572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:46069 (local address 127.27.115.254:44371)
0811 18:52:32.374741 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.374746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.375270 (+ 524us) server_negotiation.cc:407] Connection header received
0811 18:52:32.376348 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.376352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.376410 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.376496 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.377859 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.378630 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.380245 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.380815 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.381022 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.384730 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.384735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.384738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:32.384775 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.386777 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.387676 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.387680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.387682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.387803 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.388246 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.388250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.388252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.388476 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.388599 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.389487 (+ 888us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.389678 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:52:32.392025 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.373297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.194:46069)
0811 18:52:32.375075 (+ 1778us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:32.375142 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:32.376108 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:32.376712 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:32.376720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:32.377013 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:32.377646 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.377661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.378795 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.378797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.380017 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.380029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.381252 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.381258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.381515 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.382370 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:32.382398 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:32.384440 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:32.387005 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.387013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.387035 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.387527 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.387915 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.387919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.387921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.388110 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.388600 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:32.388611 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:32.389050 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 18:52:32.390221 (+ 1171us) client_negotiation.cc:240] Negotiation successful
0811 18:52:32.390698 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1543,"thread_start_us":152,"threads_started":1}
I20260811 18:52:32.393720 10833 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:32.394025 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.394829 10833 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.397008 10833 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:44371 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:52:32.397164 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.397406 10833 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:32.398461 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.398663 10833 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.399480 10833 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:52:32.559870 10836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:32.560316 10836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:32.560400 10836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:32.567044 10836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:32.567145 10836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:32.567255 10836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:52:32.575877 10836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.115.212:34443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:39157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=46581
--tserver_master_addrs=127.27.115.254:44371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:32.577509 10836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:32.579200 10836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:32.592698 10846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.592654 10843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.592690 10844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:32.593365 10836 server_base.cc:1034] running on GCE node
I20260811 18:52:32.593928 10836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:32.594908 10836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:32.596131 10836 hybrid_clock.cc:648] HybridClock initialized: now 1786474352596117 us; error 77 us; skew 500 ppm
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:32.599727 10836 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:52:32.600917 10836 webserver.cc:460] Webserver started at http://127.27.115.195:46581/ using document root <none> and password file <none>
I20260811 18:52:32.601617 10836 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:32.601739 10836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:32.624279 10836 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:52:32.627334 10854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.629089 10836 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:52:32.629297 10836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "1d55ea9b96aa41899b4f75bc3e1f1830"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-9jjr"
server_key: "2453ed35ddad9a920837c461fa931a60"
server_key_iv: "ad0e2db8be8fb3e69fa0fee6d2e52034"
server_key_version: "kuduclusterkey@0"
I20260811 18:52:32.630008 10836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:32.643221 10836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:32.647646 10836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:32.647907 10836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:32.648952 10836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:32.650192 10836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:32.650269 10836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.650347 10836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:32.650372 10836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.662235 10978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:39157 every 8 connection(s)
I20260811 18:52:32.662401 10836 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:39157
I20260811 18:52:32.663794 10836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:32.670569 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 10836
Aug 11 18:52:32 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:32.682888 10984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.671466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.27.115.254:44371)
0811 18:52:32.672179 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.672184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.672199 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:52:32.672242 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.672244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.672296 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.672402 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.673603 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.674455 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.675558 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.675932 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.676183 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.677974 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.677979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.677981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:32.678020 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.680150 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.681082 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.681087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.681089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.681193 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.681550 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.681554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.681556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.681833 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.682057 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.682311 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.682496 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":630,"spinlock_wait_cycles":4352,"thread_start_us":124,"threads_started":1}
I20260811 18:52:32.683909 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.668398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:46619 (local address 127.27.115.254:44371)
0811 18:52:32.668547 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.668552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.668573 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:52:32.669356 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.669361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.669460 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.669616 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.671794 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.672734 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.674395 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.674734 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.674848 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.678887 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.678890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.678893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:32.678954 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.680712 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.681475 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.681574 (+ 99us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.681576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.681676 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:32.682165 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:32.682169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:32.682170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:32.682430 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.682518 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.683640 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.683759 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":6400}
I20260811 18:52:32.685562 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.667109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44371 (local address 127.27.115.195:46619)
0811 18:52:32.667759 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:32.667828 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:32.669004 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:32.670198 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:32.670212 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:32.670639 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:32.671546 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.671571 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.672919 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.672923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.674260 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:32.674272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.674914 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.674919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:32.675158 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.675976 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:32.676024 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:32.678659 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:32.680884 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.680895 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.680920 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.681340 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.681789 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:32.681792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:32.681793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:32.682003 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:32.682595 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:32.682605 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:32.682892 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 18:52:32.683840 (+ 948us) client_negotiation.cc:240] Negotiation successful
0811 18:52:32.684261 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":168,"threads_started":1}
I20260811 18:52:32.686808 10980 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44371
I20260811 18:52:32.687091 10980 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.687727 10980 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.689574 10980 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:44371 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:52:32.689718 10980 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.689925 10980 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:32.690944 10980 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:32.691160 10980 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
W20260811 18:52:32.692080 10980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 18:52:33 dist-test-slave-9jjr krb5kdc[9314](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474353, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:33.150622 10674 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:33.150993 10674 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:33.152557 10450 ts_manager.cc:194] Registered new tserver with Master: c9c4020d20fa4abc914056c7605de10b (127.27.115.193:33491)
I20260811 18:52:33.156663 10450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:36695
I20260811 18:52:33.399744 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:33.400081 10833 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:33.402678 10450 ts_manager.cc:194] Registered new tserver with Master: e65c1752f6094da4b23091cafd6bbbe4 (127.27.115.194:33499)
I20260811 18:52:33.403548 10450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:46069
Aug 11 18:52:33 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474353, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:33.692318 10980 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:33.692660 10980 heartbeater.cc:511] Master 127.27.115.254:44371 requested a full tablet report, sending...
I20260811 18:52:33.693493 10450 ts_manager.cc:194] Registered new tserver with Master: 1d55ea9b96aa41899b4f75bc3e1f1830 (127.27.115.195:39157)
I20260811 18:52:33.694303 10450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:46619
I20260811 18:52:33.703171 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:33.713375 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.704880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.27.115.254:44371)
0811 18:52:33.705147 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.705151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.705391 (+ 240us) server_negotiation.cc:407] Connection header received
0811 18:52:33.705552 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.705555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.705612 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.705713 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:33.706548 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.707108 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.707948 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.708211 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.708328 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.709868 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:33.709872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:33.709875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:33.709913 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:33.711555 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:33.712017 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:33.712021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:33.712023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:33.712116 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:33.712410 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:33.712414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:33.712416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:33.712615 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:33.712773 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.713018 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.713155 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260811 18:52:33.718024 10450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:33.721868 10450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:33.745792 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.736459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:39157 (local address 127.0.0.1:32850)
0811 18:52:33.737391 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:33.737407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:33.737514 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:33.739092 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:33.739097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:33.739143 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:33.739623 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.739630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.741305 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.741309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.743537 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.743554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.745014 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.745019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.745194 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.745262 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 18:52:33.745378 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:52:33.745567 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"spinlock_wait_cycles":9728,"thread_start_us":100,"threads_started":1}
I20260811 18:52:33.746744 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.737130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.27.115.195:39157)
0811 18:52:33.738417 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.738425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.738452 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:52:33.738569 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.738577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.738865 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.739806 (+ 941us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:33.739988 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.741151 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.743756 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.744846 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.746080 (+ 1234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.746353 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.746428 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.746564 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1155,"thread_start_us":72,"threads_started":1}
I20260811 18:52:33.748730 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.735857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33499 (local address 127.0.0.1:42994)
0811 18:52:33.736546 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:33.736567 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:33.736721 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:33.739089 (+ 2368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:33.739097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:33.739142 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:33.739364 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.739378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.742714 (+ 3336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.742719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.744036 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.744048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.748204 (+ 4156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.748208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.748315 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.748344 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:52:33.748454 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:52:33.748566 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":19,"spinlock_wait_cycles":6912,"thread_start_us":139,"threads_started":1}
I20260811 18:52:33.748755 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.736939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.27.115.194:33499)
0811 18:52:33.737854 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.737864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.737894 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:52:33.738685 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.738690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.738865 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.739238 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:33.739530 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.742480 (+ 2950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.744391 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.747668 (+ 3277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.748207 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.748375 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.748488 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.748612 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":110,"threads_started":1}
I20260811 18:52:33.750684 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8d362e196242baab504c65ab983397 (DEFAULT_TABLE table=test-table [id=eb3200284f2347b7b545012f9febc1d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:33.752084 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8d362e196242baab504c65ab983397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:33.753039 10740 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8d362e196242baab504c65ab983397 (DEFAULT_TABLE table=test-table [id=eb3200284f2347b7b545012f9febc1d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:33.755242 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8d362e196242baab504c65ab983397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:33.757623 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.734042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:33491 (local address 127.0.0.1:39374)
0811 18:52:33.734550 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:33.734666 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:33.734889 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:33.747184 (+ 12295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:33.747188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:33.747218 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:33.747358 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.747365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.754036 (+ 6671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.754042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.755235 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.755255 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.757055 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.757059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.757181 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.757214 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:52:33.757328 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 18:52:33.757436 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":196,"threads_started":1}
I20260811 18:52:33.758234 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.734196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.27.115.193:33491)
0811 18:52:33.746492 (+ 12296us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.746501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.746537 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:52:33.746639 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.746646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.746957 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.747557 (+ 600us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:33.750243 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.751485 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.755484 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.756583 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.756768 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.756948 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.757948 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.758064 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12132,"thread_start_us":99,"threads_started":1}
I20260811 18:52:33.761570 10600 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8d362e196242baab504c65ab983397 (DEFAULT_TABLE table=test-table [id=eb3200284f2347b7b545012f9febc1d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:33.762852 10600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8d362e196242baab504c65ab983397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:33.775003 11017 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Bootstrap starting.
I20260811 18:52:33.777895 11017 tablet_bootstrap.cc:654] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:33.779034 11017 log.cc:824] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:33.780169 11018 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Bootstrap starting.
I20260811 18:52:33.783134 11018 tablet_bootstrap.cc:654] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:33.784251 11018 log.cc:824] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:33.787904 11018 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: No bootstrap required, opened a new log
I20260811 18:52:33.788218 11018 ts_tablet_manager.cc:1397] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 18:52:33.790794 11017 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: No bootstrap required, opened a new log
I20260811 18:52:33.791136 11017 ts_tablet_manager.cc:1397] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.003s
I20260811 18:52:33.793299 11018 raft_consensus.cc:348] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.793803 11018 raft_consensus.cc:374] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:33.793871 11018 raft_consensus.cc:729] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c4020d20fa4abc914056c7605de10b, State: Initialized, Role: FOLLOWER
I20260811 18:52:33.794561 11018 consensus_queue.cc:260] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.796231 11018 ts_tablet_manager.cc:1428] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 18:52:33.796747 10674 heartbeater.cc:503] Master 127.27.115.254:44371 was elected leader, sending a full tablet report...
I20260811 18:52:33.799201 11017 raft_consensus.cc:348] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.799719 11017 raft_consensus.cc:374] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:33.799909 11017 raft_consensus.cc:729] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1752f6094da4b23091cafd6bbbe4, State: Initialized, Role: FOLLOWER
I20260811 18:52:33.800665 11017 consensus_queue.cc:260] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.802281 11017 ts_tablet_manager.cc:1428] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Time spent starting tablet: real 0.011s user 0.006s sys 0.002s
I20260811 18:52:33.802816 10833 heartbeater.cc:503] Master 127.27.115.254:44371 was elected leader, sending a full tablet report...
I20260811 18:52:33.803459 11021 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Bootstrap starting.
I20260811 18:52:33.807094 11021 tablet_bootstrap.cc:654] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:33.808287 11021 log.cc:824] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:33.811723 11021 tablet_bootstrap.cc:492] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: No bootstrap required, opened a new log
I20260811 18:52:33.811995 11021 ts_tablet_manager.cc:1397] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 18:52:33.816352 11021 raft_consensus.cc:348] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.816758 11021 raft_consensus.cc:374] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:33.816838 11021 raft_consensus.cc:729] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d55ea9b96aa41899b4f75bc3e1f1830, State: Initialized, Role: FOLLOWER
I20260811 18:52:33.817440 11021 consensus_queue.cc:260] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.818444 10980 heartbeater.cc:503] Master 127.27.115.254:44371 was elected leader, sending a full tablet report...
I20260811 18:52:33.819818 11021 ts_tablet_manager.cc:1428] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
W20260811 18:52:33.858981 10675 tablet.cc:2369] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:33.872430 10834 tablet.cc:2369] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:52:33 dist-test-slave-9jjr krb5kdc[9314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474353, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for HTTP/127.27.115.212@KRBTEST.COM
I20260811 18:52:33.896066 11025 raft_consensus.cc:482] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:33.896389 11025 raft_consensus.cc:504] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.897962 11025 leader_election.cc:288] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9c4020d20fa4abc914056c7605de10b (127.27.115.193:33491), e65c1752f6094da4b23091cafd6bbbe4 (127.27.115.194:33499)
I20260811 18:52:33.905236 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.898507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:38249 (local address 127.27.115.193:33491)
0811 18:52:33.898643 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.898647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.899046 (+ 399us) server_negotiation.cc:407] Connection header received
0811 18:52:33.899504 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.899509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.899564 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.899614 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:33.900084 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.900850 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.903501 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.904165 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.904273 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.904331 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.904961 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.905068 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:52:33.905787 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.898263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:33491 (local address 127.27.115.195:38249)
0811 18:52:33.898961 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:33.898973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:33.899362 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:33.899763 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:33.899766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:33.899785 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:33.899941 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.899948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.901288 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.901291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.903304 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.903314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.904672 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.904676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.904776 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.904819 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:52:33.905515 (+ 696us) client_negotiation.cc:240] Negotiation successful
0811 18:52:33.905611 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":310,"thread_start_us":122,"threads_started":1}
I20260811 18:52:33.905922 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.898362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:47097 (local address 127.27.115.194:33499)
0811 18:52:33.898505 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.898510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.899045 (+ 535us) server_negotiation.cc:407] Connection header received
0811 18:52:33.899294 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.899299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.899362 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.899416 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:33.902158 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.902964 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.904562 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.905181 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.905327 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.905380 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.905702 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.905787 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:52:33.906340 11028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.898259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33499 (local address 127.27.115.195:47097)
0811 18:52:33.898939 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:33.898955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:33.899145 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:33.901816 (+ 2671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:33.901819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:33.901844 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:33.901963 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.901970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.903170 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.903173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.904380 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:33.904393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.905447 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.905451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:33.905558 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.905583 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:52:33.906085 (+ 502us) client_negotiation.cc:240] Negotiation successful
0811 18:52:33.906194 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":141,"threads_started":1}
I20260811 18:52:33.906704 10620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8d362e196242baab504c65ab983397" candidate_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c4020d20fa4abc914056c7605de10b" is_pre_election: true
I20260811 18:52:33.907112 10620 raft_consensus.cc:2454] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d55ea9b96aa41899b4f75bc3e1f1830 in term 0.
I20260811 18:52:33.907724 10858 leader_election.cc:302] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d55ea9b96aa41899b4f75bc3e1f1830, c9c4020d20fa4abc914056c7605de10b; no voters:
I20260811 18:52:33.907850 10772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8d362e196242baab504c65ab983397" candidate_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1752f6094da4b23091cafd6bbbe4" is_pre_election: true
I20260811 18:52:33.908077 11025 raft_consensus.cc:2781] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:33.908128 10772 raft_consensus.cc:2454] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d55ea9b96aa41899b4f75bc3e1f1830 in term 0.
I20260811 18:52:33.908190 11025 raft_consensus.cc:482] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:33.908239 11025 raft_consensus.cc:3037] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:33.909675 11025 raft_consensus.cc:504] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
I20260811 18:52:33.910120 11025 leader_election.cc:288] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [CANDIDATE]: Term 1 election: Requested vote from peers c9c4020d20fa4abc914056c7605de10b (127.27.115.193:33491), e65c1752f6094da4b23091cafd6bbbe4 (127.27.115.194:33499)
I20260811 18:52:33.910619 10772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8d362e196242baab504c65ab983397" candidate_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1752f6094da4b23091cafd6bbbe4"
I20260811 18:52:33.910666 10620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8d362e196242baab504c65ab983397" candidate_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c4020d20fa4abc914056c7605de10b"
I20260811 18:52:33.910769 10772 raft_consensus.cc:3037] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:33.910816 10620 raft_consensus.cc:3037] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:33.912225 10772 raft_consensus.cc:2454] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d55ea9b96aa41899b4f75bc3e1f1830 in term 1.
I20260811 18:52:33.912227 10620 raft_consensus.cc:2454] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d55ea9b96aa41899b4f75bc3e1f1830 in term 1.
I20260811 18:52:33.912654 10858 leader_election.cc:302] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d55ea9b96aa41899b4f75bc3e1f1830, c9c4020d20fa4abc914056c7605de10b; no voters:
I20260811 18:52:33.912884 11025 raft_consensus.cc:2781] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:33.913138 11025 raft_consensus.cc:686] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 1 LEADER]: Becoming Leader. State: Replica: 1d55ea9b96aa41899b4f75bc3e1f1830, State: Running, Role: LEADER
I20260811 18:52:33.913628 11025 consensus_queue.cc:237] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } }
W20260811 18:52:33.915825 10981 tablet.cc:2369] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:33.918166 10450 catalog_manager.cc:5495] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d55ea9b96aa41899b4f75bc3e1f1830 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d55ea9b96aa41899b4f75bc3e1f1830" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 39157 } health_report { overall_health: HEALTHY } } }
I20260811 18:52:33.944676 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:33.940417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.27.115.195:39157)
0811 18:52:33.940531 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:33.940536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:33.940549 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:52:33.940647 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:33.940650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:33.940732 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:33.940872 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:33.941161 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.941689 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.942817 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:33.943166 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:33.943284 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:33.943645 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:33.944099 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:33.944249 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:33.944390 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 18:52:33.944488 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:52:33.952118 10772 raft_consensus.cc:1261] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 1 FOLLOWER]: Refusing update from remote peer 1d55ea9b96aa41899b4f75bc3e1f1830: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:33.952153 10620 raft_consensus.cc:1261] T 0b8d362e196242baab504c65ab983397 P c9c4020d20fa4abc914056c7605de10b [term 1 FOLLOWER]: Refusing update from remote peer 1d55ea9b96aa41899b4f75bc3e1f1830: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:33.952988 11025 consensus_queue.cc:1035] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65c1752f6094da4b23091cafd6bbbe4" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:33.953212 11029 consensus_queue.cc:1035] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9c4020d20fa4abc914056c7605de10b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 33491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:33.970674 11034 mvcc.cc:205] Tried to move back new op lower bound from 7317398953779081216 to 7317398953634193408. Current Snapshot: MvccSnapshot[applied={T|T < 7317398953779081216}]
I20260811 18:52:33.978523 10450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:33.978744 10450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:33.982403 10450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecab900f9df641ed878c1d00477f2947: Sending DeleteTablet for 3 replicas of tablet 0b8d362e196242baab504c65ab983397
I20260811 18:52:33.984016 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b8d362e196242baab504c65ab983397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:33 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:32850
I20260811 18:52:33.984681 10600 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b8d362e196242baab504c65ab983397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:33 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:39374
I20260811 18:52:33.986188 10740 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b8d362e196242baab504c65ab983397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:33 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:42994
I20260811 18:52:33.988220 11036 mvcc.cc:205] Tried to move back new op lower bound from 7317398953779081216 to 7317398953634193408. Current Snapshot: MvccSnapshot[applied={T|T < 7317398953779081216}]
I20260811 18:52:33.989490 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10493
I20260811 18:52:33.995410 11044 tablet_replica.cc:330] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: stopping tablet replica
I20260811 18:52:33.995934 11044 raft_consensus.cc:2229] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:33.996306 11044 raft_consensus.cc:2258] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:34.008268 11042 tablet_replica.cc:330] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: stopping tablet replica
I20260811 18:52:34.008859 11042 raft_consensus.cc:2229] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:52:34.009619 11042 raft_consensus.cc:2258] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:34.019930 11042 ts_tablet_manager.cc:1905] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:34.023077 11044 ts_tablet_manager.cc:1905] T 0b8d362e196242baab504c65ab983397 P e65c1752f6094da4b23091cafd6bbbe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:34.023137 11042 ts_tablet_manager.cc:1918] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:34.023245 11042 log.cc:1196] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/0b8d362e196242baab504c65ab983397
I20260811 18:52:34.023602 11042 ts_tablet_manager.cc:1939] T 0b8d362e196242baab504c65ab983397 P 1d55ea9b96aa41899b4f75bc3e1f1830: Deleting consensus metadata
I20260811 18:52:34.025033 10435 catalog_manager.cc:4841] TS 1d55ea9b96aa41899b4f75bc3e1f1830 (127.27.115.195:39157): tablet 0b8d362e196242baab504c65ab983397 (table test-table [id=eb3200284f2347b7b545012f9febc1d0]) successfully deleted
W20260811 18:52:34.025594 10437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:33491 (error 108)
I20260811 18:52:34.027132 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10678
I20260811 18:52:34.027197 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:34.026850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:33491 (local address 127.0.0.1:39390)
0811 18:52:34.027003 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:34.027100 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:33491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 18:52:34.034898 10437 catalog_manager.cc:4584] TS c9c4020d20fa4abc914056c7605de10b (127.27.115.193:33491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b8d362e196242baab504c65ab983397: Network error: Client connection negotiation failed: client connection to 127.27.115.193:33491: connect: Connection refused (error 111)
W20260811 18:52:34.051170 11011 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:34.050464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:33499 (local address 127.0.0.1:43006)
0811 18:52:34.050671 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:34.050687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:34.050814 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:34.051034 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.194:33499: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 18:52:34.051889 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10836
I20260811 18:52:34.062861 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 10414
I20260811 18:52:34.073688 28111 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:52:34.419329 28111 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:52:34.419483 28111 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:52:34.765571 28111 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:52:34.765 UTC [9348] LOG: received smart shutdown request
2026-08-11 18:52:34.770 UTC [9348] LOG: background worker "logical replication launcher" (PID 9357) exited with exit code 1
2026-08-11 18:52:34.770 UTC [9352] LOG: shutting down
2026-08-11 18:52:34.961 UTC [9348] LOG: database system is shut down
2026-08-11T18:52:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55297 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:35 dist-test-slave-9jjr krb5kdc[11058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:35 dist-test-slave-9jjr krb5kdc[11058](info): set up 2 sockets
Aug 11 18:52:35 dist-test-slave-9jjr krb5kdc[11058](info): commencing operation
krb5kdc: starting...
W20260811 18:52:36.210382 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:52:36.355784 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39873
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:36.473632 11073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:36.473999 11073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:36.474083 11073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:36.478297 11073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:36.478399 11073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:36.478439 11073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:36.478477 11073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:36.478513 11073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:36.484094 11073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:36.485378 11073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:36.486562 11073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:36.495867 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.495859 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.495869 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.496163 11073 server_base.cc:1034] running on GCE node
I20260811 18:52:36.496881 11073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:36.498070 11073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:36.499313 11073 hybrid_clock.cc:648] HybridClock initialized: now 1786474356499297 us; error 30 us; skew 500 ppm
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:36.502435 11073 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:36.503441 11073 webserver.cc:460] Webserver started at http://127.27.115.254:42573/ using document root <none> and password file <none>
I20260811 18:52:36.504026 11073 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:36.504122 11073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:36.504377 11073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:36.506556 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "cd7bbd7495714113a3b11b146f34ea8e"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4328dfbddcddd9a67209aed5bc454873"
server_key_iv: "efdd11e1027cae78af7f3f639b8e4dec"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.507157 11073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "cd7bbd7495714113a3b11b146f34ea8e"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4328dfbddcddd9a67209aed5bc454873"
server_key_iv: "efdd11e1027cae78af7f3f639b8e4dec"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.511077 11073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:52:36.513698 11089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:36.515218 11073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:36.515404 11073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "cd7bbd7495714113a3b11b146f34ea8e"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4328dfbddcddd9a67209aed5bc454873"
server_key_iv: "efdd11e1027cae78af7f3f639b8e4dec"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.515532 11073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:36.531652 11073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:36.535203 11073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:36.535429 11073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:36.543172 11073 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39873
I20260811 18:52:36.543459 11149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39873 every 8 connection(s)
I20260811 18:52:36.544200 11073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:36.547324 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:36.551159 11150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.554006 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11073
I20260811 18:52:36.554171 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:52:36.554458 28111 external_mini_cluster.cc:1428] Setting key 6902f597f6f7f38c582384ff966f6259
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.253@KRBTEST.COM" created.
I20260811 18:52:36.567883 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.553454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39873 (local address 127.0.0.1:60156)
0811 18:52:36.553955 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.554002 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.555083 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.555753 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.555759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.556217 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.556455 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.556468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.557656 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.557664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.558483 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.558493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.559171 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.559175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.559271 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.560023 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.560043 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.561916 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.564121 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.564130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.564139 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.564452 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.564907 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.564910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.564911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.565072 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.565953 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.565959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.566278 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.566528 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.566898 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
I20260811 18:52:36.567885 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.554429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.27.115.254:39873)
0811 18:52:36.554764 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.554774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.554820 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:52:36.555384 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.555388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.555561 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.555754 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.556614 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.557505 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.558621 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.559000 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.559192 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.562082 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.562089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.562101 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.562169 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.563933 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.564595 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.564601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.564602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.564757 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.565265 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.565289 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.565294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.565800 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.565987 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.566617 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.567031 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":222,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260811 18:52:36.570896 11150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.572013 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.571570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60654)
0811 18:52:36.571760 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.571895 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 18:52:36.572398 11094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
I20260811 18:52:36.573469 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.573219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60660)
0811 18:52:36.573345 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.573397 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:52:36.573746 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33055: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
I20260811 18:52:36.586484 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:33055
--webserver_interface=127.27.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:36.623111 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } attempt: 1
I20260811 18:52:36.624246 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.623855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60672)
0811 18:52:36.624061 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.624140 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 18:52:36.625046 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.624821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60680)
0811 18:52:36.624933 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.624988 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 18:52:36.625192 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33055: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
I20260811 18:52:36.693363 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } attempt: 2
I20260811 18:52:36.694370 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.693976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60688)
0811 18:52:36.694196 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.694288 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:52:36.695138 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.694911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60698)
0811 18:52:36.695024 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.695079 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:52:36.695281 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33055: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
W20260811 18:52:36.715058 11157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:36.715431 11157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:36.715525 11157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:36.720011 11157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:36.720126 11157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:36.720166 11157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:36.720208 11157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:36.720240 11157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:36.726415 11157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:36.727705 11157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:36.728875 11157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:36.738907 11162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.738921 11165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.739073 11157 server_base.cc:1034] running on GCE node
W20260811 18:52:36.739305 11167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.740072 11157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:36.741596 11157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:36.742848 11157 hybrid_clock.cc:648] HybridClock initialized: now 1786474356742832 us; error 46 us; skew 500 ppm
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:36.747139 11157 init.cc:377] Logged in from keytab as kudu/127.27.115.253@KRBTEST.COM (short username kudu)
I20260811 18:52:36.748360 11157 webserver.cc:460] Webserver started at http://127.27.115.253:41511/ using document root <none> and password file <none>
I20260811 18:52:36.749032 11157 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:36.749145 11157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:36.749423 11157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:36.751979 11157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data/instance:
uuid: "b799acd33d164ea7abf595d21ce7e8ec"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "d355edfa737b3138301d583cfaec560b"
server_key_iv: "7cd96ad21d5fa457683be136d845834a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.752720 11157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal/instance:
uuid: "b799acd33d164ea7abf595d21ce7e8ec"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "d355edfa737b3138301d583cfaec560b"
server_key_iv: "7cd96ad21d5fa457683be136d845834a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.757318 11157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 18:52:36.760169 11174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:36.761515 11157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:52:36.761680 11157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "b799acd33d164ea7abf595d21ce7e8ec"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "d355edfa737b3138301d583cfaec560b"
server_key_iv: "7cd96ad21d5fa457683be136d845834a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.761816 11157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:36.778538 11157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:36.781931 11157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:36.782194 11157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:36.786459 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } attempt: 3
I20260811 18:52:36.787524 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.787118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60706)
0811 18:52:36.787315 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.787404 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 18:52:36.788478 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.788193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60708)
0811 18:52:36.788351 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.788406 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 18:52:36.788760 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33055: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33055: connect: Connection refused (error 111)
I20260811 18:52:36.790525 11157 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.253:33055
I20260811 18:52:36.790562 11230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.253:33055 every 8 connection(s)
I20260811 18:52:36.791668 11157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
I20260811 18:52:36.795085 11231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:36.795557 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11157
I20260811 18:52:36.795702 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-1/wal/instance
I20260811 18:52:36.796022 28111 external_mini_cluster.cc:1428] Setting key f97fc7d059511b121a377216d0c67c21
I20260811 18:52:36.800339 11231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.115.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.252@KRBTEST.COM" created.
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:36.816697 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.802658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.27.115.254:39873)
0811 18:52:36.802819 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.802823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.804087 (+ 1264us) server_negotiation.cc:407] Connection header received
0811 18:52:36.805119 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.805122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.805190 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.805306 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.806768 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.807331 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.808662 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.808973 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.809096 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.812130 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.812134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.812136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.812168 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.813772 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.814570 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.814574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.814575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.814667 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.815227 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.815230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.815231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.815453 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.815536 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.816426 (+ 890us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.816558 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:52:36.818174 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.803290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39873 (local address 127.0.0.1:60160)
0811 18:52:36.803911 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.803960 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.804888 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.805528 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.805536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.805902 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.806600 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.806618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.807484 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.807487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.808518 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.808550 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.809215 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.809219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.809400 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.810236 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.810256 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.811897 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.813930 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.813940 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.813961 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.814441 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.814785 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.814791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.814796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.815041 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.815584 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.815594 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.816058 (+ 464us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.816317 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.816772 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":174,"threads_started":1}
I20260811 18:52:36.820211 11231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
I20260811 18:52:36.831235 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:43807
--webserver_interface=127.27.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:36.832196 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.821025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60714)
0811 18:52:36.821177 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.821194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.821309 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.822178 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.822182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.822407 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.822665 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.822672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.823796 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.823799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.824366 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.824375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.825292 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.825296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.825386 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.826028 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.826042 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.827776 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.830267 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.830274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.830285 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.830569 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.830966 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.830968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.830969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.831051 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.831719 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.831721 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.831816 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.831903 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.832024 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"spinlock_wait_cycles":278784}
I20260811 18:52:36.832506 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.821229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.27.115.253:33055)
0811 18:52:36.821685 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.821724 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.821748 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:52:36.821791 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.821796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.821980 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.822164 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.822791 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.823655 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.824536 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.825084 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.825230 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.827958 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.827972 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.827989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.828015 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.830102 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.830705 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.830715 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.830717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.830856 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.831150 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.831156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.831158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.831583 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.831741 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.832045 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.832237 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":206,"threads_started":1}
I20260811 18:52:36.834623 11231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.835592 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.835192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43378)
0811 18:52:36.835365 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.835473 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 18:52:36.835865 11175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
I20260811 18:52:36.836966 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.836643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43380)
0811 18:52:36.836799 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.836871 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:52:36.837139 11231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:43807: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
I20260811 18:52:36.886512 11231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } attempt: 1
I20260811 18:52:36.887682 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.887248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43386)
0811 18:52:36.887473 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.887564 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 18:52:36.888626 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.888353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43402)
0811 18:52:36.888477 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.888541 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 18:52:36.888852 11231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:43807: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
I20260811 18:52:36.931989 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } attempt: 4
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:52:36.944561 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.932703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60722)
0811 18:52:36.933000 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.933022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.933145 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.933443 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.933446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.933687 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.933831 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.933840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.935692 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.935696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.937352 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.937373 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.938065 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.938069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.938269 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.938930 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.938949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.940516 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.942816 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.942819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.942821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.943089 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.943440 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.943443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.943444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.943533 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.944048 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.944051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.944160 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.944264 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.944402 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260811 18:52:36.944687 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.932854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.27.115.253:33055)
0811 18:52:36.933018 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.933022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.933103 (+ 81us) server_negotiation.cc:407] Connection header received
0811 18:52:36.933247 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.933250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.933301 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.933390 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.934023 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.935383 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.937598 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.937892 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.938013 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.940740 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.940748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.940751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.940790 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.942646 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.943223 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.943227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.943229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.943334 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.943628 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.943632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.943633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.943900 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.944021 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.944357 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.944515 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:52:36.945650 11150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.946504 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.946248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43410)
0811 18:52:36.946373 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.946437 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 18:52:36.947270 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.947061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43422)
0811 18:52:36.947168 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.947219 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:52:36.947528 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:43807: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
I20260811 18:52:36.957098 11231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } attempt: 2
I20260811 18:52:36.958122 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.957751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43436)
0811 18:52:36.957945 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.958028 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 18:52:36.959209 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.958815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43438)
0811 18:52:36.959039 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.959137 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260811 18:52:36.959347 11231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:43807: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
W20260811 18:52:36.965618 11237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:36.966003 11237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:36.966102 11237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:36.970645 11237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:36.970759 11237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:36.970796 11237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:36.970850 11237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:36.970903 11237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:36.976989 11237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39873,127.27.115.253:33055,127.27.115.252:43807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:43807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:36.978370 11237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:36.979487 11237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:36.989079 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.989079 11243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.989079 11244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.989490 11237 server_base.cc:1034] running on GCE node
I20260811 18:52:36.990321 11237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:36.991664 11237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:36.992903 11237 hybrid_clock.cc:648] HybridClock initialized: now 1786474356992869 us; error 49 us; skew 500 ppm
Aug 11 18:52:36 dist-test-slave-9jjr krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:36.996331 11237 init.cc:377] Logged in from keytab as kudu/127.27.115.252@KRBTEST.COM (short username kudu)
I20260811 18:52:36.997481 11237 webserver.cc:460] Webserver started at http://127.27.115.252:41419/ using document root <none> and password file <none>
I20260811 18:52:36.998251 11237 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:36.998366 11237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:36.998637 11237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:37.001077 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data/instance:
uuid: "90894af56d8846da8c413a1d9b4e02af"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4e94882ed0227509e74f4b46a1aa8260"
server_key_iv: "c8d904a16376455a55e2b898ff7445c8"
server_key_version: "encryptionkey@0"
I20260811 18:52:37.001777 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal/instance:
uuid: "90894af56d8846da8c413a1d9b4e02af"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4e94882ed0227509e74f4b46a1aa8260"
server_key_iv: "c8d904a16376455a55e2b898ff7445c8"
server_key_version: "encryptionkey@0"
I20260811 18:52:37.006485 11237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:52:37.006742 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } attempt: 1
I20260811 18:52:37.007618 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.007285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43440)
0811 18:52:37.007464 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.007531 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 18:52:37.008446 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.008205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43452)
0811 18:52:37.008326 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.008385 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:52:37.008647 11150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:43807: Network error: Client connection negotiation failed: client connection to 127.27.115.252:43807: connect: Connection refused (error 111)
I20260811 18:52:37.009862 11253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:37.011677 11237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:37.011843 11237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "90894af56d8846da8c413a1d9b4e02af"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-9jjr"
server_key: "4e94882ed0227509e74f4b46a1aa8260"
server_key_iv: "c8d904a16376455a55e2b898ff7445c8"
server_key_version: "encryptionkey@0"
I20260811 18:52:37.011966 11237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:37.032352 11237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:37.036100 11237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:37.036316 11237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:37.044176 11237 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.252:43807
I20260811 18:52:37.044515 11317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.252:43807 every 8 connection(s)
I20260811 18:52:37.045264 11237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
I20260811 18:52:37.048345 11318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:37.050524 11231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } attempt: 3
I20260811 18:52:37.051014 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11237
I20260811 18:52:37.051157 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474173680671-28111-0/minicluster-data/master-2/wal/instance
I20260811 18:52:37.051478 28111 external_mini_cluster.cc:1428] Setting key 64bea204fa085f23cd65616c8b80a84a
I20260811 18:52:37.053100 11318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:37.066861 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.051141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43460)
0811 18:52:37.051412 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.051430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:37.051539 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:37.054029 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:37.054032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:37.054301 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:37.054460 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.054471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.056477 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.056481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.057049 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.057062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.057947 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.057954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.058044 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.058775 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:37.058796 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:37.060417 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:37.063878 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.063884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.063887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.064188 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.064590 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.064594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.064595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.064711 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.066223 (+ 1512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:37.066226 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:37.066385 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 18:52:37.066494 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:52:37.066652 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 18:52:37.067600 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.055408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.27.115.254:39873)
0811 18:52:37.055542 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.055547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.055896 (+ 349us) server_negotiation.cc:407] Connection header received
0811 18:52:37.056112 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.056115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.056167 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.056256 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.057087 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.057793 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.059091 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.059451 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.059584 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.062745 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.062749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.062752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.062786 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.064511 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.065186 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.065189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.065191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.065346 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.065844 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.065872 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.065874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.066226 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.066334 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.067104 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.067299 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:52:37.068459 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.055345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39873 (local address 127.0.0.1:60168)
0811 18:52:37.055772 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.055822 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:37.055985 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:37.056306 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:37.056312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:37.056778 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:37.056928 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.056938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.057934 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.057937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.058941 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.058957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.059644 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.059649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.059774 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.060347 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:37.060367 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:37.062503 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:37.064647 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.064660 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.064667 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.065006 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.065530 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.065536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.065541 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.065697 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.066572 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:37.066579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:37.066763 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 18:52:37.066963 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 18:52:37.067374 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":104,"threads_started":1}
I20260811 18:52:37.068472 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.051992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.27.115.252:43807)
0811 18:52:37.052600 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.052626 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.052679 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:52:37.053348 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.053365 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.053719 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.054083 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.054664 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.056290 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.057172 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.057797 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.057992 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.060590 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.060600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.060620 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.060648 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.063472 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.064320 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.064334 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.064341 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.064489 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.064817 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.064820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.064822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.065504 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.065659 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.067223 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.067446 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":528,"server-negotiator.queue_time_us":439,"spinlock_wait_cycles":15360,"thread_start_us":155,"threads_started":1}
I20260811 18:52:37.068472 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.052465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.27.115.252:43807)
0811 18:52:37.052763 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.052771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.052788 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:52:37.053388 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.053393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.053722 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.054082 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.055050 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.056290 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.057454 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.057808 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.058343 (+ 535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.061310 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.061314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.061317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.061349 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.063924 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.064521 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.064527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.064531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.064625 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.065025 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.065051 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.065056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.065503 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.065732 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.067934 (+ 2202us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.068068 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":259,"server-negotiator.queue_time_us":231,"spinlock_wait_cycles":30208,"thread_start_us":91,"threads_started":1}
I20260811 18:52:37.070474 11318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:37.073323 11231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: Bootstrap starting.
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:52:37.075821 11150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } attempt: 2
I20260811 18:52:37.076330 11231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:37.077201 11231 log.cc:824] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:37.080013 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.071532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.27.115.253:33055)
0811 18:52:37.071663 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.071666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.071692 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:52:37.071728 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.071731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.071768 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.071824 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.072386 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.072869 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.073845 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.074049 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.074188 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.076425 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.076429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.076431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.076464 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.078324 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.078766 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.078771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.078773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.078856 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.079145 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.079149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.079151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.079358 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.079473 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.079768 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.079881 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:52:37.080020 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.071290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33055 (local address 127.0.0.1:60734)
0811 18:52:37.071526 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.071539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:37.071627 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:37.071867 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:37.071870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:37.072071 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:37.072260 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.072271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.072980 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.072983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.073727 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.073738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.074190 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.074194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.074286 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.074784 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:37.074805 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:37.076247 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:37.078442 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.078446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.078448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.078661 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.078953 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.078957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.078959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.079050 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.079462 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:37.079465 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:37.079574 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 18:52:37.079687 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:52:37.079838 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 18:52:37.080123 11231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: No bootstrap required, opened a new log
I20260811 18:52:37.081346 11318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:37.085753 11231 raft_consensus.cc:348] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.086053 11231 raft_consensus.cc:374] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:37.086107 11231 raft_consensus.cc:729] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b799acd33d164ea7abf595d21ce7e8ec, State: Initialized, Role: FOLLOWER
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
I20260811 18:52:37.087248 11231 consensus_queue.cc:260] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.087845 11325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.088056 11325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
I20260811 18:52:37.088945 11231 sys_catalog.cc:564] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:37.090282 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.077752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60170 (local address 127.27.115.254:39873)
0811 18:52:37.077877 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.077882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.077895 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:52:37.077937 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.077940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.077984 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.078065 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.079030 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.079717 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.080777 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.081086 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.081209 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.084275 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.084279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.084282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.084312 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.086364 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.086915 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.086919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.086921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.087019 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.087367 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.087370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.087372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.088930 (+ 1558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.089167 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.089919 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.090056 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50,"spinlock_wait_cycles":70528}
I20260811 18:52:37.091825 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.081879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43494)
0811 18:52:37.081981 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.081995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:37.082068 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:37.082530 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:37.082533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:37.082704 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:37.082782 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.082788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.083850 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.083853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.084511 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.084520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.085128 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.085132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.085206 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.085864 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:37.085876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:37.087611 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:37.090070 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.090072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.090074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.090339 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.090626 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.090631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.090633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.090723 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.091297 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:37.091299 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:37.091398 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 18:52:37.091506 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:52:37.091650 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:52:37.092685 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.081913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.27.115.252:43807)
0811 18:52:37.082301 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.082306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.082322 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:52:37.082361 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.082365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.082414 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.082503 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.082905 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.083732 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.084628 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.084998 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.085126 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.087749 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.087753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.087755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.087777 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.089961 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.090447 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.090449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.090451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.090529 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.090816 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.090821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.090823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.091153 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.091262 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.092095 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.092525 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
I20260811 18:52:37.094308 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.076321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:43807 (local address 127.0.0.1:43478)
0811 18:52:37.076465 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:37.076481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:37.076588 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:37.078113 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:37.078115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:37.078345 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:37.078449 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.078460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.083081 (+ 4621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.083085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.083875 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:37.083886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.086837 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.086840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:37.086933 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.087403 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:37.087439 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:37.088943 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:37.091650 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.091656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.091660 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.091939 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.092742 (+ 803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:37.092745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:37.092747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:37.093305 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:37.093751 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:37.093754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:37.093856 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 18:52:37.093948 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:52:37.094097 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"spinlock_wait_cycles":14208}
I20260811 18:52:37.095163 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.076521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43478 (local address 127.27.115.252:43807)
0811 18:52:37.077903 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.077909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.077921 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:52:37.077952 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.077954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.077990 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.082204 (+ 4214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.082294 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.082964 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.086444 (+ 3480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.086718 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.086832 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.089129 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.089133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.089135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.089171 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.091494 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.092378 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.092381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.092384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.092479 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.093425 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.093428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.093430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.093633 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.094778 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.094862 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.094979 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309}
I20260811 18:52:37.097172 11318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: Bootstrap starting.
Aug 11 18:52:37 dist-test-slave-9jjr krb5kdc[11058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:52:37.094254 11157 master_runner.cc:186] Error getting master registration for 127.27.115.254:39873: OK, instance_id { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" instance_seqno: 1786474356541161 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:52:37.098713 11157 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:52:37.098824 11150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: Bootstrap starting.
I20260811 18:52:37.100587 11318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:37.101389 11150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:37.101603 11318 log.cc:824] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:37.102023 11150 log.cc:824] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:37.103462 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.091799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.27.115.253:33055)
0811 18:52:37.091922 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.091928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.092097 (+ 169us) server_negotiation.cc:407] Connection header received
0811 18:52:37.092297 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.092300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.092351 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.092404 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.093354 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.093896 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.095167 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.095668 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.095768 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.099134 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.099138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.099141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.099181 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.101032 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.101645 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.101650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.101651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.101754 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.102122 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.102177 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.102181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.102434 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.102646 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.103144 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.103306 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:52:37.104735 11150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: No bootstrap required, opened a new log
I20260811 18:52:37.105286 11318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: No bootstrap required, opened a new log
W20260811 18:52:37.106920 11337 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:37.107062 11337 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:37.107717 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:37.109941 11150 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.110275 11150 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:37.110342 11150 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7bbd7495714113a3b11b146f34ea8e, State: Initialized, Role: FOLLOWER
I20260811 18:52:37.110872 11150 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.112484 11150 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:37.112543 11318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.113041 11318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:37.113148 11318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90894af56d8846da8c413a1d9b4e02af, State: Initialized, Role: FOLLOWER
I20260811 18:52:37.113766 11318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.115628 11318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:37.123833 11073 master_runner.cc:263] No leader master found from master cd7bbd7495714113a3b11b146f34ea8e
I20260811 18:52:37.123939 11073 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:52:37.124174 11355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.124323 11355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.125468 11237 master_runner.cc:263] No leader master found from master 90894af56d8846da8c413a1d9b4e02af
I20260811 18:52:37.125550 11237 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:52:37.125761 11362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:37.125851 11362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:37.132241 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.110302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.27.115.252:43807)
0811 18:52:37.110527 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.110532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.114598 (+ 4066us) server_negotiation.cc:407] Connection header received
0811 18:52:37.114901 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.114905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.114963 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.115023 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.121803 (+ 6780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.122571 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.125208 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.125634 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.125768 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.127862 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.127867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.127871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.127904 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.129743 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.130492 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.130496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.130498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.130593 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.130998 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.131002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.131005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.131285 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.131406 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.131828 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.131993 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 18:52:37.132265 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.109909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.27.115.254:39873)
0811 18:52:37.110052 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.110058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.114603 (+ 4545us) server_negotiation.cc:407] Connection header received
0811 18:52:37.114870 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.114875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.114932 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.114995 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.116389 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.117222 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.118684 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.119008 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.119090 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.126228 (+ 7138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.126234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.126237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.126277 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.128253 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.130746 (+ 2493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.130750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.130752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.130853 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.131215 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.131219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.131221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.131462 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.131590 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.131948 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.132076 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 18:52:37.132095 11375 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:37.132558 11375 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:37.134343 11349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.134516 11349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.142084 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.110622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.27.115.253:33055)
0811 18:52:37.110803 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.110808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.127340 (+ 16532us) server_negotiation.cc:407] Connection header received
0811 18:52:37.128543 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.128548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.128622 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.128686 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.129721 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.130601 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.135534 (+ 4933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.135888 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.135976 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.138049 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:37.138054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:37.138057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:37.138090 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:37.139937 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.140491 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.140496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.140498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.140593 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:37.140938 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:37.140941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:37.140943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:37.141225 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:37.141348 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.141759 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.141903 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260811 18:52:37.284034 11180 tablet.cc:2369] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:37.287360 11096 tablet.cc:2369] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:37.287798 11262 tablet.cc:2369] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:37.455614 11325 raft_consensus.cc:482] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:37.455911 11325 raft_consensus.cc:504] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.457101 11325 leader_election.cc:288] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd7bbd7495714113a3b11b146f34ea8e (127.27.115.254:39873), 90894af56d8846da8c413a1d9b4e02af (127.27.115.252:43807)
I20260811 18:52:37.457852 11285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b799acd33d164ea7abf595d21ce7e8ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90894af56d8846da8c413a1d9b4e02af" is_pre_election: true
I20260811 18:52:37.457854 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b799acd33d164ea7abf595d21ce7e8ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7bbd7495714113a3b11b146f34ea8e" is_pre_election: true
I20260811 18:52:37.458272 11121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b799acd33d164ea7abf595d21ce7e8ec in term 0.
I20260811 18:52:37.458277 11285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b799acd33d164ea7abf595d21ce7e8ec in term 0.
I20260811 18:52:37.458887 11177 leader_election.cc:302] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 90894af56d8846da8c413a1d9b4e02af, b799acd33d164ea7abf595d21ce7e8ec, cd7bbd7495714113a3b11b146f34ea8e; no voters:
I20260811 18:52:37.459322 11325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:37.459410 11325 raft_consensus.cc:482] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:37.459442 11325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:37.460670 11325 raft_consensus.cc:504] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.461124 11325 leader_election.cc:288] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [CANDIDATE]: Term 1 election: Requested vote from peers cd7bbd7495714113a3b11b146f34ea8e (127.27.115.254:39873), 90894af56d8846da8c413a1d9b4e02af (127.27.115.252:43807)
I20260811 18:52:37.461634 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b799acd33d164ea7abf595d21ce7e8ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7bbd7495714113a3b11b146f34ea8e"
I20260811 18:52:37.461666 11285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b799acd33d164ea7abf595d21ce7e8ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90894af56d8846da8c413a1d9b4e02af"
I20260811 18:52:37.461796 11121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:37.461823 11285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:37.463246 11285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b799acd33d164ea7abf595d21ce7e8ec in term 1.
I20260811 18:52:37.463258 11121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b799acd33d164ea7abf595d21ce7e8ec in term 1.
I20260811 18:52:37.463701 11175 leader_election.cc:302] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90894af56d8846da8c413a1d9b4e02af, b799acd33d164ea7abf595d21ce7e8ec; no voters:
I20260811 18:52:37.463997 11325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:37.464239 11325 raft_consensus.cc:686] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [term 1 LEADER]: Becoming Leader. State: Replica: b799acd33d164ea7abf595d21ce7e8ec, State: Running, Role: LEADER
I20260811 18:52:37.464601 11325 consensus_queue.cc:237] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } }
I20260811 18:52:37.467010 11379 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader b799acd33d164ea7abf595d21ce7e8ec. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.467224 11379 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:37.467582 11381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:37.468477 11381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:37.473565 11121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [term 1 FOLLOWER]: Refusing update from remote peer b799acd33d164ea7abf595d21ce7e8ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:37.473611 11285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [term 1 FOLLOWER]: Refusing update from remote peer b799acd33d164ea7abf595d21ce7e8ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:37.474442 11325 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:37.474653 11379 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:37.478931 11355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: SysCatalogTable state changed. Reason: New leader b799acd33d164ea7abf595d21ce7e8ec. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.478940 11349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b799acd33d164ea7abf595d21ce7e8ec. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.479090 11349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.479094 11355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.482080 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.482158 11355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.482291 11355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.482401 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:37.482457 11349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.482575 11349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:37.482700 11325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b799acd33d164ea7abf595d21ce7e8ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7bbd7495714113a3b11b146f34ea8e" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39873 } } peers { permanent_uuid: "b799acd33d164ea7abf595d21ce7e8ec" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33055 } } peers { permanent_uuid: "90894af56d8846da8c413a1d9b4e02af" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 43807 } } }
I20260811 18:52:37.482805 11325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:37.482916 11385 mvcc.cc:205] Tried to move back new op lower bound from 7317398968203681792 to 7317398968177844224. Current Snapshot: MvccSnapshot[applied={T|T < 7317398968203681792}]
I20260811 18:52:37.483938 11381 catalog_manager.cc:1347] Generated new cluster ID: ed037e7274bf4b72b470b9baf2cc383c
I20260811 18:52:37.484009 11381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:37.516597 11381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:37.518078 11381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:37.527730 11381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b799acd33d164ea7abf595d21ce7e8ec: Generated new TSK 0
I20260811 18:52:37.528664 11381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:38.127389 11362 catalog_manager.cc:1259] Loaded cluster ID: ed037e7274bf4b72b470b9baf2cc383c
I20260811 18:52:38.127491 11362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: loading cluster ID for follower catalog manager: success
I20260811 18:52:38.129315 11362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: acquiring CA information for follower catalog manager: success
I20260811 18:52:38.130749 11362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd7bbd7495714113a3b11b146f34ea8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:52:38.133955 11375 catalog_manager.cc:1259] Loaded cluster ID: ed037e7274bf4b72b470b9baf2cc383c
I20260811 18:52:38.134042 11375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: loading cluster ID for follower catalog manager: success
I20260811 18:52:38.135762 11375 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: acquiring CA information for follower catalog manager: success
I20260811 18:52:38.136931 11375 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90894af56d8846da8c413a1d9b4e02af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:52:38.604359 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11073
I20260811 18:52:38.614593 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11157
I20260811 18:52:38.624719 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11237
2026-08-11T18:52:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3537 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:38 dist-test-slave-9jjr krb5kdc[11402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:38 dist-test-slave-9jjr krb5kdc[11402](info): set up 2 sockets
Aug 11 18:52:38 dist-test-slave-9jjr krb5kdc[11402](info): commencing operation
krb5kdc: starting...
W20260811 18:52:39.756699 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:52:39 dist-test-slave-9jjr krb5kdc[11402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:52:39.902736 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35401
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:35401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:40.022723 11416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:40.023094 11416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:40.023140 11416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:40.027617 11416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:40.027720 11416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:40.027741 11416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:40.027758 11416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:40.027776 11416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:40.033892 11416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:35401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:40.035256 11416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:40.036429 11416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:40.045879 11423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:40.046051 11416 server_base.cc:1034] running on GCE node
W20260811 18:52:40.045905 11422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:40.045879 11425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:40.047026 11416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:40.048285 11416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:40.049523 11416 hybrid_clock.cc:648] HybridClock initialized: now 1786474360049501 us; error 38 us; skew 500 ppm
Aug 11 18:52:40 dist-test-slave-9jjr krb5kdc[11402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474360, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:40.052876 11416 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:40.054018 11416 webserver.cc:460] Webserver started at http://127.27.115.254:39857/ using document root <none> and password file <none>
I20260811 18:52:40.054705 11416 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:40.054778 11416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:40.054976 11416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:40.057260 11416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "98cb98d1c2e9425c8e5ad3352cea5998"
format_stamp: "Formatted at 2026-08-11 18:52:40 on dist-test-slave-9jjr"
server_key: "bd63ef87c383c00a547c504b4818e702"
server_key_iv: "48fdc3fd0587aeae73972e64b338c7cb"
server_key_version: "encryptionkey@0"
I20260811 18:52:40.057847 11416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "98cb98d1c2e9425c8e5ad3352cea5998"
format_stamp: "Formatted at 2026-08-11 18:52:40 on dist-test-slave-9jjr"
server_key: "bd63ef87c383c00a547c504b4818e702"
server_key_iv: "48fdc3fd0587aeae73972e64b338c7cb"
server_key_version: "encryptionkey@0"
I20260811 18:52:40.062290 11416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260811 18:52:40.065285 11433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:40.066900 11416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:52:40.067042 11416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "98cb98d1c2e9425c8e5ad3352cea5998"
format_stamp: "Formatted at 2026-08-11 18:52:40 on dist-test-slave-9jjr"
server_key: "bd63ef87c383c00a547c504b4818e702"
server_key_iv: "48fdc3fd0587aeae73972e64b338c7cb"
server_key_version: "encryptionkey@0"
I20260811 18:52:40.067173 11416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:40.090186 11416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:40.093868 11416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:40.094058 11416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:40.102227 11416 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:35401
I20260811 18:52:40.102249 11495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:35401 every 8 connection(s)
I20260811 18:52:40.103317 11416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:40.106391 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:40.111615 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11416
I20260811 18:52:40.111760 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:52:40.112067 28111 external_mini_cluster.cc:1428] Setting key 9749c5ade9a9ea207e567a616232cd28
I20260811 18:52:40.113022 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: Bootstrap starting.
I20260811 18:52:40.115896 11496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:40.116788 11496 log.cc:824] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:40.119535 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: No bootstrap required, opened a new log
Aug 11 18:52:40 dist-test-slave-9jjr krb5kdc[11402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:40.123999 11496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } }
I20260811 18:52:40.124279 11496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:40.124367 11496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cb98d1c2e9425c8e5ad3352cea5998, State: Initialized, Role: FOLLOWER
I20260811 18:52:40.124889 11496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } }
I20260811 18:52:40.125025 11496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:40.125116 11496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:40.125243 11496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:40.126407 11496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } }
I20260811 18:52:40.126806 11496 leader_election.cc:302] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98cb98d1c2e9425c8e5ad3352cea5998; no voters:
I20260811 18:52:40.127207 11496 leader_election.cc:288] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:40.127308 11502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:40.127525 11502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [term 1 LEADER]: Becoming Leader. State: Replica: 98cb98d1c2e9425c8e5ad3352cea5998, State: Running, Role: LEADER
I20260811 18:52:40.127923 11502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } }
I20260811 18:52:40.128890 11496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:40.129522 11502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98cb98d1c2e9425c8e5ad3352cea5998. Latest consensus state: current_term: 1 leader_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } } }
I20260811 18:52:40.129743 11502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:40.129891 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.113700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.27.115.254:35401)
0811 18:52:40.114383 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.114400 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.114444 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:52:40.115203 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.115224 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.115630 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.116054 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:40.117155 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.118582 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.119571 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.120199 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.120408 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.123525 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:40.123532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:40.123560 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:40.123600 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:40.125846 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:40.126467 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:40.126475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:40.126483 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:40.126660 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:40.126989 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:40.126992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:40.126994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:40.127593 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:40.128433 (+ 840us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.128567 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.128837 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":507,"thread_start_us":172,"threads_started":1}
I20260811 18:52:40.129940 11503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb98d1c2e9425c8e5ad3352cea5998" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35401 } } }
I20260811 18:52:40.130028 11503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:40.133644 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 18:52:40.135067 11518 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:40.135187 11518 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:40.135258 11511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:40.136240 11511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:40.143865 11511 catalog_manager.cc:1347] Generated new cluster ID: e05362d7ed9444ae8db68f77b50ef045
I20260811 18:52:40.143996 11511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:40.162374 11511 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:40.163841 11511 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:40.170917 11511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98cb98d1c2e9425c8e5ad3352cea5998: Generated new TSK 0
I20260811 18:52:40.171896 11511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:40.190454 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11416
2026-08-11T18:52:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1562 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:40 dist-test-slave-9jjr krb5kdc[11528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:40 dist-test-slave-9jjr krb5kdc[11528](info): set up 2 sockets
Aug 11 18:52:40 dist-test-slave-9jjr krb5kdc[11528](info): commencing operation
krb5kdc: starting...
W20260811 18:52:41.355654 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:52:41 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:52:41.501808 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:41.623785 11542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:41.624104 11542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:41.624156 11542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:41.628463 11542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:41.628561 11542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:41.628580 11542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:41.628598 11542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:41.628613 11542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:41.634457 11542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:41.635732 11542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:41.636879 11542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:41.647322 11548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:41.647357 11550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:41.647328 11547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:41.647661 11542 server_base.cc:1034] running on GCE node
I20260811 18:52:41.648468 11542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:41.649930 11542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:41.651193 11542 hybrid_clock.cc:648] HybridClock initialized: now 1786474361651175 us; error 42 us; skew 500 ppm
Aug 11 18:52:41 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474361, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:41.654788 11542 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:41.656167 11542 webserver.cc:460] Webserver started at http://127.27.115.254:46569/ using document root <none> and password file <none>
I20260811 18:52:41.656947 11542 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:41.657061 11542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:41.657346 11542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:41.660053 11542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "39042c5814fa4b07b7dd7bb710bde14a"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "f47c570e0484e689631450541a32cab2"
server_key_iv: "d0a7ecb7035eb1ccb01b4d7e3c4d4508"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.660832 11542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "39042c5814fa4b07b7dd7bb710bde14a"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "f47c570e0484e689631450541a32cab2"
server_key_iv: "d0a7ecb7035eb1ccb01b4d7e3c4d4508"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.665943 11542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:52:41.669281 11557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:41.670915 11542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:41.671062 11542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "39042c5814fa4b07b7dd7bb710bde14a"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "f47c570e0484e689631450541a32cab2"
server_key_iv: "d0a7ecb7035eb1ccb01b4d7e3c4d4508"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.671191 11542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:41.690374 11542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:41.693841 11542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:41.694020 11542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:41.701867 11542 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39939
I20260811 18:52:41.702221 11619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39939 every 8 connection(s)
I20260811 18:52:41.703047 11542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:41.706292 11620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:41.710654 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11542
I20260811 18:52:41.710858 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:52:41.711166 28111 external_mini_cluster.cc:1428] Setting key de567d242eaecca3493e7a7e3018e098
I20260811 18:52:41.713085 11620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap starting.
I20260811 18:52:41.716063 11620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:41.716936 11620 log.cc:824] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:41.719705 11620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: No bootstrap required, opened a new log
Aug 11 18:52:41 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:41.723790 11620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:41.724172 11620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:41.724277 11620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Initialized, Role: FOLLOWER
I20260811 18:52:41.724803 11620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:41.724977 11620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:41.725073 11620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:41.725203 11620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:41.726406 11620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:41.726817 11620 leader_election.cc:302] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39042c5814fa4b07b7dd7bb710bde14a; no voters:
I20260811 18:52:41.727274 11620 leader_election.cc:288] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:41.727880 11627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:41.728122 11627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 LEADER]: Becoming Leader. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Running, Role: LEADER
I20260811 18:52:41.728539 11627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:41.728899 11620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:41.729076 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:41.712811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49058 (local address 127.27.115.254:39939)
0811 18:52:41.713328 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:41.713345 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:41.713384 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:52:41.714091 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:41.714109 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:41.714552 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:41.714979 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:41.715930 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:41.717253 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:41.718382 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:41.718887 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:41.719057 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:41.722031 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:41.722038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:41.722054 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:41.722096 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:41.725041 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:41.725681 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:41.725689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:41.725696 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:41.725855 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:41.726234 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:41.726238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:41.726240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:41.726814 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:41.727067 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:41.727413 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 18:52:41.727778 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":168,"threads_started":1}
I20260811 18:52:41.731221 11626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39042c5814fa4b07b7dd7bb710bde14a. Latest consensus state: current_term: 1 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:41.731668 11626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:41.731223 11628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:41.732064 11628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:41.734382 11640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:41.734489 11640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:41.734582 11643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:41.735575 11643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:41.743129 11643 catalog_manager.cc:1347] Generated new cluster ID: 277a48617dac498ebea4d1ec6193f995
I20260811 18:52:41.743338 11643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:41.768893 11643 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:41.770058 11643 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:41.787804 11643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Generated new TSK 0
I20260811 18:52:41.788733 11643 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:52:41.827033 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:41.952986 11649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:41.953266 11649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:41.953318 11649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:41.957372 11649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:41.957438 11649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:41.957497 11649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:52:41.962862 11649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:41.964051 11649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:41.965098 11649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:41.975208 11657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:41.975208 11654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:41.975212 11655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:41.975580 11649 server_base.cc:1034] running on GCE node
I20260811 18:52:41.976017 11649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:41.976774 11649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:41.977970 11649 hybrid_clock.cc:648] HybridClock initialized: now 1786474361977921 us; error 62 us; skew 500 ppm
Aug 11 18:52:41 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474361, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:41.981107 11649 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:52:41.981999 11649 webserver.cc:460] Webserver started at http://127.27.115.193:37273/ using document root <none> and password file <none>
I20260811 18:52:41.982605 11649 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:41.982669 11649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:41.982851 11649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:41.984923 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "8b9fb725e466aa1b56af45ee372c9ff5"
server_key_iv: "d2f56144c5ab6d598c5a3ac92befbb2a"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.985544 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "8b9fb725e466aa1b56af45ee372c9ff5"
server_key_iv: "d2f56144c5ab6d598c5a3ac92befbb2a"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.989384 11649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:52:41.991959 11666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:41.993427 11649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:52:41.993600 11649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "8b9fb725e466aa1b56af45ee372c9ff5"
server_key_iv: "d2f56144c5ab6d598c5a3ac92befbb2a"
server_key_version: "encryptionkey@0"
I20260811 18:52:41.993726 11649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:42.007619 11649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:42.010991 11649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:42.011219 11649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:42.011862 11649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:42.012740 11649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:42.012827 11649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.012920 11649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:42.012972 11649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.022910 11649 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37477
I20260811 18:52:42.022933 11794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37477 every 8 connection(s)
I20260811 18:52:42.024065 11649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:42.025964 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11649
I20260811 18:52:42.026134 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:42.026463 28111 external_mini_cluster.cc:1428] Setting key a1b59d0fce4c80317c856fc41d06b5df
Aug 11 18:52:42 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474361, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:52:42.041033 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.026201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:51861 (local address 127.27.115.254:39939)
0811 18:52:42.026356 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.026362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.027430 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 18:52:42.028929 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.028935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.029002 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.029098 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:42.030679 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.031351 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.033213 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.033497 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.033633 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.036859 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:42.036863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:42.036865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:42.036893 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:42.038611 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.039278 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.039287 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.039288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.039378 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.039774 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.039778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.039780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.039998 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:42.040111 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.040699 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.040850 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:52:42.042110 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.026729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:51861)
0811 18:52:42.027264 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.027322 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.028695 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.029267 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.029274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.029785 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:42.030506 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.030521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.031495 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.031502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.033053 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.033069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.033724 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.033731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.033981 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.034697 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:42.034726 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:42.036640 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:42.038792 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.038800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.038828 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.039144 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.039482 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.039485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.039487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.039661 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.040127 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:42.040136 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:42.040436 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.040641 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.041029 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":155,"threads_started":1}
I20260811 18:52:42.043607 11795 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:42.043885 11795 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:42.044430 11795 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
I20260811 18:52:42.046258 11575 ts_manager.cc:194] Registered new tserver with Master: e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477)
I20260811 18:52:42.048014 11575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:51861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:52:42.061480 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:42.181989 11801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:42.182432 11801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:42.182514 11801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:42.186960 11801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:42.187052 11801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:42.187148 11801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:52:42.193004 11801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:42.194360 11801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:42.195472 11801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:42.206370 11812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:42.206370 11809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:42.206372 11808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:42.206418 11801 server_base.cc:1034] running on GCE node
I20260811 18:52:42.207219 11801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:42.207993 11801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:42.209197 11801 hybrid_clock.cc:648] HybridClock initialized: now 1786474362209164 us; error 38 us; skew 500 ppm
Aug 11 18:52:42 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:42.212659 11801 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:52:42.213969 11801 webserver.cc:460] Webserver started at http://127.27.115.194:33969/ using document root <none> and password file <none>
I20260811 18:52:42.214740 11801 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:42.214826 11801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:42.215133 11801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:42.217628 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "b719659bc0694944a78c64f5587b3988"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "16da74d4efd3e918e7dde1f377f2ab7a"
server_key_iv: "a1eee4efa1a63044e77e5ad010f14bc1"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.218340 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "b719659bc0694944a78c64f5587b3988"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "16da74d4efd3e918e7dde1f377f2ab7a"
server_key_iv: "a1eee4efa1a63044e77e5ad010f14bc1"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.222826 11801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:52:42.225693 11820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.227228 11801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:42.227406 11801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b719659bc0694944a78c64f5587b3988"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "16da74d4efd3e918e7dde1f377f2ab7a"
server_key_iv: "a1eee4efa1a63044e77e5ad010f14bc1"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.227543 11801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:42.244529 11801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:42.248306 11801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:42.248554 11801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:42.249253 11801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:42.250298 11801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:42.250351 11801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.250419 11801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:42.250458 11801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.260682 11801 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:34497
I20260811 18:52:42.260877 11951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:34497 every 8 connection(s)
I20260811 18:52:42.261754 11801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:42.270222 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11801
I20260811 18:52:42.270390 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:42.270735 28111 external_mini_cluster.cc:1428] Setting key 3cf05efec5f9c332cdf7cbd95dd88150
Aug 11 18:52:42 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:42.278414 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.263971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:40701 (local address 127.27.115.254:39939)
0811 18:52:42.264181 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.264185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.265099 (+ 914us) server_negotiation.cc:407] Connection header received
0811 18:52:42.266699 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.266702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.266770 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.266887 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:42.268274 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.268817 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.270086 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.270508 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.270656 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.274018 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:42.274022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:42.274024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:42.274051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:42.275843 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.276436 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.276440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.276442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.276572 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.276973 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.276977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.276978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.277202 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:42.277319 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.277981 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.278191 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:52:42.279265 11954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.264408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:40701)
0811 18:52:42.264920 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.264969 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.266435 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.267093 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.267107 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.267534 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:42.268070 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.268085 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.268987 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.268990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.269936 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.269947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.270787 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.270795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.271171 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.272003 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:42.272022 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:42.273809 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:42.275979 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.275986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.276013 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.276318 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.276691 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.276695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.276697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.276864 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.277322 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:42.277332 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:42.277635 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.277880 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.278277 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":120,"threads_started":1}
I20260811 18:52:42.280794 11952 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:42.281102 11952 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:42.281704 11952 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:52:42.283094 11575 ts_manager.cc:194] Registered new tserver with Master: b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497)
I20260811 18:52:42.283824 11575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:40701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:52:42.306066 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:42.432812 11958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:42.433197 11958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:42.433290 11958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:42.437500 11958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:42.437626 11958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:42.437736 11958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:52:42.443392 11958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:42.444675 11958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:42.445757 11958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:42.456920 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:42.456938 11965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:42.457231 11958 server_base.cc:1034] running on GCE node
W20260811 18:52:42.456920 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:42.457866 11958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:42.458689 11958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:42.460000 11958 hybrid_clock.cc:648] HybridClock initialized: now 1786474362459933 us; error 79 us; skew 500 ppm
Aug 11 18:52:42 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:42.463399 11958 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:52:42.464397 11958 webserver.cc:460] Webserver started at http://127.27.115.195:33937/ using document root <none> and password file <none>
I20260811 18:52:42.464993 11958 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:42.465063 11958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:42.465257 11958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:42.467554 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "e328205ecddb4f8eb415d05469a37c5f"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "b69dea988b399eb6612bbc013da920c6"
server_key_iv: "7f445b35e56fd5b7df73378ea8514103"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.468165 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "e328205ecddb4f8eb415d05469a37c5f"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "b69dea988b399eb6612bbc013da920c6"
server_key_iv: "7f445b35e56fd5b7df73378ea8514103"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.472476 11958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:52:42.475764 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.477497 11958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:52:42.477711 11958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "e328205ecddb4f8eb415d05469a37c5f"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "b69dea988b399eb6612bbc013da920c6"
server_key_iv: "7f445b35e56fd5b7df73378ea8514103"
server_key_version: "encryptionkey@0"
I20260811 18:52:42.477861 11958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:42.509495 11958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:42.513204 11958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:42.513475 11958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:42.514295 11958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:42.515311 11958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:42.515369 11958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.515413 11958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:42.515429 11958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:42.525899 11958 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33873
I20260811 18:52:42.526068 12103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33873 every 8 connection(s)
I20260811 18:52:42.527220 11958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:42.535833 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 11958
I20260811 18:52:42.535990 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:42.536372 28111 external_mini_cluster.cc:1428] Setting key 9cb7c0b2a113b49c4b01962b17830aec
Aug 11 18:52:42 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474362, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:42.545625 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.529705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:39209 (local address 127.27.115.254:39939)
0811 18:52:42.529894 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.529901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.531393 (+ 1492us) server_negotiation.cc:407] Connection header received
0811 18:52:42.533266 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.533270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.533350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.533479 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:42.535053 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.535873 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.538175 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.538533 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.538690 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.541930 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:42.541933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:42.541936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:42.541968 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:42.543540 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.544139 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.544142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.544143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.544221 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.544552 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.544554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.544555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.544749 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:42.544837 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.545378 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.545498 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:52:42.546633 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.530343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:39209)
0811 18:52:42.531130 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.531243 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.532910 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.533725 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.533734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.534234 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:42.534851 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.534870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.536156 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.536178 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.537931 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.537950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.538784 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.538790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.539018 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.539852 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:42.539873 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:42.541701 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:42.543693 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.543700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.543716 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.544039 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.544308 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:42.544310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:42.544311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:42.544463 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:42.544852 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:42.544858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:42.545131 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.545318 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.545608 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":280,"threads_started":1}
I20260811 18:52:42.547979 12105 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:42.548270 12105 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:42.548883 12105 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
I20260811 18:52:42.550280 11575 ts_manager.cc:194] Registered new tserver with Master: e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
I20260811 18:52:42.551030 11575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:39209
I20260811 18:52:42.551597 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:42.561682 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.553773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.27.115.254:39939)
0811 18:52:42.553942 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.553947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.554083 (+ 136us) server_negotiation.cc:407] Connection header received
0811 18:52:42.554269 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.554272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.554332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.554421 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:42.555174 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.555708 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.556602 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.556844 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.556957 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.558450 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:42.558453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:42.558456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:42.558487 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:42.559854 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.560297 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.560304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.560307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.560398 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:42.560683 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:42.560687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:42.560689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:42.560951 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:42.561101 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.561381 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.561514 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:52:42.565279 11574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:42.568552 11574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:42.584388 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.578605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.27.115.194:34497)
0811 18:52:42.579759 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.579772 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.579795 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:52:42.579867 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.579872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.580049 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.580499 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:42.580657 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581677 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.583031 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.583635 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.583782 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.583942 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.584112 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.584210 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":141,"threads_started":1}
I20260811 18:52:42.584373 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.578408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:34497 (local address 127.0.0.1:53464)
0811 18:52:42.579408 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.579420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.579490 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.580199 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.580205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.580228 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:42.580346 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.580352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581823 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.581829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.582907 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.582917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.583770 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.583774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.583907 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.583949 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.584073 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.584210 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":87,"threads_started":1}
I20260811 18:52:42.584718 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.578880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.27.115.193:37477)
0811 18:52:42.579282 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.579292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.579323 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:52:42.579406 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.579413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.579643 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.579805 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:42.580160 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581401 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.582884 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.583732 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.583880 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.584020 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.584456 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.584563 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":98,"threads_started":1}
I20260811 18:52:42.584847 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.578016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.0.0.1:44562)
0811 18:52:42.578958 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.579014 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.579151 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.579829 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.579835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.579861 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:42.580005 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.580012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581542 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.581546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.582754 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.582764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.584083 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.584087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.584183 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.584208 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.584280 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.584352 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":27,"thread_start_us":134,"threads_started":1}
I20260811 18:52:42.585860 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.578086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33873 (local address 127.0.0.1:33480)
0811 18:52:42.579238 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.579251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.579332 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.580050 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.580053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.580078 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:42.580218 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.580225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581267 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.581272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.583776 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.583791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.585447 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.585450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.585556 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.585581 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.585659 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.585736 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":28,"thread_start_us":124,"threads_started":1}
I20260811 18:52:42.586396 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.579022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.27.115.195:33873)
0811 18:52:42.579601 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.579613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.579635 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:52:42.579724 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.579732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.579895 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.580069 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:42.580340 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.581127 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.583924 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.585121 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.585258 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.585380 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.586110 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.586256 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":132,"threads_started":1}
I20260811 18:52:42.587702 11721 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad (DEFAULT_TABLE table=test-table [id=f27578d425e34892ae251ed71b4cd6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:42.587692 11873 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad (DEFAULT_TABLE table=test-table [id=f27578d425e34892ae251ed71b4cd6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:42.588735 11873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa7822e5a11431e9b03e12bb01df7ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:42.588778 11721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa7822e5a11431e9b03e12bb01df7ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:42.591450 12033 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad (DEFAULT_TABLE table=test-table [id=f27578d425e34892ae251ed71b4cd6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:42.592831 12033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa7822e5a11431e9b03e12bb01df7ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:42.596063 12125 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap starting.
I20260811 18:52:42.598752 12125 tablet_bootstrap.cc:654] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:42.598755 12126 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Bootstrap starting.
I20260811 18:52:42.599833 12125 log.cc:824] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:42.601485 12128 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Bootstrap starting.
I20260811 18:52:42.601716 12126 tablet_bootstrap.cc:654] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:42.602491 12125 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: No bootstrap required, opened a new log
I20260811 18:52:42.602658 12126 log.cc:824] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:42.602741 12125 ts_tablet_manager.cc:1397] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260811 18:52:42.605020 12126 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: No bootstrap required, opened a new log
I20260811 18:52:42.605124 12128 tablet_bootstrap.cc:654] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:42.605258 12126 ts_tablet_manager.cc:1397] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:52:42.606106 12128 log.cc:824] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:42.607893 12125 raft_consensus.cc:348] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.608209 12125 raft_consensus.cc:374] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:42.608286 12125 raft_consensus.cc:729] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bd57379d2f4fd5ae6e295a62c0c2b3, State: Initialized, Role: FOLLOWER
I20260811 18:52:42.608618 12128 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: No bootstrap required, opened a new log
I20260811 18:52:42.608840 12128 ts_tablet_manager.cc:1397] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260811 18:52:42.608798 12125 consensus_queue.cc:260] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.609066 12126 raft_consensus.cc:348] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.609346 12126 raft_consensus.cc:374] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:42.609417 12126 raft_consensus.cc:729] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b719659bc0694944a78c64f5587b3988, State: Initialized, Role: FOLLOWER
I20260811 18:52:42.609733 11795 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:42.609912 12126 consensus_queue.cc:260] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.610843 12126 ts_tablet_manager.cc:1428] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 18:52:42.611217 11952 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:42.612871 12128 raft_consensus.cc:348] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.613169 12128 raft_consensus.cc:374] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:42.613241 12128 raft_consensus.cc:729] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328205ecddb4f8eb415d05469a37c5f, State: Initialized, Role: FOLLOWER
I20260811 18:52:42.613714 12128 consensus_queue.cc:260] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.613797 12125 ts_tablet_manager.cc:1428] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent starting tablet: real 0.011s user 0.000s sys 0.008s
I20260811 18:52:42.614540 12105 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:42.614856 12128 ts_tablet_manager.cc:1428] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 18:52:42.735808 12133 raft_consensus.cc:482] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:42.736104 12133 raft_consensus.cc:504] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.737383 12133 leader_election.cc:288] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
I20260811 18:52:42.742662 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.737808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:53903 (local address 127.27.115.193:37477)
0811 18:52:42.737982 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.737989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.738375 (+ 386us) server_negotiation.cc:407] Connection header received
0811 18:52:42.738481 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.738484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.738524 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.738623 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:42.738881 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.739605 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.741208 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.741885 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.742029 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.742196 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.742423 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.742511 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:52:42.742662 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.737800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:54687 (local address 127.27.115.195:33873)
0811 18:52:42.737988 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.737993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.738012 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:52:42.738074 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.738077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.738179 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.738261 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:42.738637 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.739422 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.741208 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.741885 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.742029 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.742091 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.742425 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.742510 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:52:42.742707 11954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.737653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33873 (local address 127.27.115.194:54687)
0811 18:52:42.737811 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.737829 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.737945 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.738307 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.738312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.738339 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:42.738494 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.738508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.739580 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.739583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.741035 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.741049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.742060 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.742064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.742235 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.742304 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.742387 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.742509 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":87,"spinlock_wait_cycles":1920}
I20260811 18:52:42.742717 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.737667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.27.115.194:53903)
0811 18:52:42.738252 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.738276 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:42.738398 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:42.738656 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:42.738658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:42.738674 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:42.738766 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.738771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.739719 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.739722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.741035 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:42.741049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.742060 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.742064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:42.742238 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.742296 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:52:42.742392 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:52:42.742514 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":255,"spinlock_wait_cycles":2304,"thread_start_us":108,"threads_started":1}
I20260811 18:52:42.743558 11744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa7822e5a11431e9b03e12bb01df7ad" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" is_pre_election: true
I20260811 18:52:42.743558 12054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa7822e5a11431e9b03e12bb01df7ad" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e328205ecddb4f8eb415d05469a37c5f" is_pre_election: true
I20260811 18:52:42.743949 11744 raft_consensus.cc:2454] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 0.
I20260811 18:52:42.743949 12054 raft_consensus.cc:2454] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 0.
I20260811 18:52:42.744585 11824 leader_election.cc:302] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f, e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters:
I20260811 18:52:42.744900 12133 raft_consensus.cc:2781] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:42.744992 12133 raft_consensus.cc:482] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:42.745036 12133 raft_consensus.cc:3037] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:42.746273 12133 raft_consensus.cc:504] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.746685 12133 leader_election.cc:288] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 election: Requested vote from peers e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
I20260811 18:52:42.747159 11744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa7822e5a11431e9b03e12bb01df7ad" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
I20260811 18:52:42.747308 11744 raft_consensus.cc:3037] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:42.747365 12054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa7822e5a11431e9b03e12bb01df7ad" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e328205ecddb4f8eb415d05469a37c5f"
I20260811 18:52:42.747560 12054 raft_consensus.cc:3037] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:42.748564 11744 raft_consensus.cc:2454] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 1.
I20260811 18:52:42.748783 12054 raft_consensus.cc:2454] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 1.
I20260811 18:52:42.749102 11824 leader_election.cc:302] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b719659bc0694944a78c64f5587b3988, e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters:
I20260811 18:52:42.749333 12133 raft_consensus.cc:2781] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:42.749640 12133 raft_consensus.cc:686] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 1 LEADER]: Becoming Leader. State: Replica: b719659bc0694944a78c64f5587b3988, State: Running, Role: LEADER
I20260811 18:52:42.750087 12133 consensus_queue.cc:237] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:42.754349 11574 catalog_manager.cc:5495] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 reported cstate change: term changed from 0 to 1, leader changed from <none> to b719659bc0694944a78c64f5587b3988 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "b719659bc0694944a78c64f5587b3988" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } health_report { overall_health: HEALTHY } } }
W20260811 18:52:42.763361 11953 tablet.cc:2369] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:42.775650 11796 tablet.cc:2369] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:42.778462 12107 tablet.cc:2369] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:42.783690 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.779472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.27.115.194:34497)
0811 18:52:42.779634 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:42.779638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:42.779654 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:52:42.779789 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:42.779792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:42.779852 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:42.780044 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:42.780312 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.780827 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.781805 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:42.782033 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:42.782174 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:42.782588 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:42.783123 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:42.783227 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:42.783411 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 18:52:42.783509 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:52:42.791025 12054 raft_consensus.cc:1261] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Refusing update from remote peer b719659bc0694944a78c64f5587b3988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:42.791026 11744 raft_consensus.cc:1261] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Refusing update from remote peer b719659bc0694944a78c64f5587b3988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:42.791896 12133 consensus_queue.cc:1035] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:42.792263 12138 consensus_queue.cc:1035] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:42.799820 12141 mvcc.cc:205] Tried to move back new op lower bound from 7317398989982187520 to 7317398989828501504. Current Snapshot: MvccSnapshot[applied={T|T < 7317398989982187520}]
I20260811 18:52:42.802659 12142 mvcc.cc:205] Tried to move back new op lower bound from 7317398989982187520 to 7317398989828501504. Current Snapshot: MvccSnapshot[applied={T|T < 7317398989982187520}]
I20260811 18:52:42.811028 11575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:42.811277 11575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:42.814450 11575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Sending DeleteTablet for 3 replicas of tablet 7aa7822e5a11431e9b03e12bb01df7ad
I20260811 18:52:42.815554 12033 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:33480
I20260811 18:52:42.815555 11721 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:44562
I20260811 18:52:42.815665 11873 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:53464
I20260811 18:52:42.816272 12154 tablet_replica.cc:330] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: stopping tablet replica
I20260811 18:52:42.816288 12151 tablet_replica.cc:330] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: stopping tablet replica
I20260811 18:52:42.816334 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11649
I20260811 18:52:42.816694 12151 raft_consensus.cc:2229] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:42.816716 12154 raft_consensus.cc:2229] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:52:42.817130 12151 raft_consensus.cc:2258] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:42.817196 12154 raft_consensus.cc:2258] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:42.819886 12154 ts_tablet_manager.cc:1905] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:42.824137 12154 ts_tablet_manager.cc:1918] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:42.824293 12154 log.cc:1196] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/7aa7822e5a11431e9b03e12bb01df7ad
I20260811 18:52:42.824717 12154 ts_tablet_manager.cc:1939] T 7aa7822e5a11431e9b03e12bb01df7ad P b719659bc0694944a78c64f5587b3988: Deleting consensus metadata
I20260811 18:52:42.827080 11560 catalog_manager.cc:4841] TS b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): tablet 7aa7822e5a11431e9b03e12bb01df7ad (table test-table [id=f27578d425e34892ae251ed71b4cd6f3]) successfully deleted
I20260811 18:52:42.831815 12151 ts_tablet_manager.cc:1905] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:52:42.833217 11561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:37477 (error 108)
I20260811 18:52:42.833503 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11801
I20260811 18:52:42.834714 12151 ts_tablet_manager.cc:1918] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:42.834817 12151 log.cc:1196] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/7aa7822e5a11431e9b03e12bb01df7ad
I20260811 18:52:42.835160 12151 ts_tablet_manager.cc:1939] T 7aa7822e5a11431e9b03e12bb01df7ad P e328205ecddb4f8eb415d05469a37c5f: Deleting consensus metadata
I20260811 18:52:42.837225 11559 catalog_manager.cc:4841] TS e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): tablet 7aa7822e5a11431e9b03e12bb01df7ad (table test-table [id=f27578d425e34892ae251ed71b4cd6f3]) successfully deleted
I20260811 18:52:42.837734 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:42.836988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.0.0.1:44564)
0811 18:52:42.837540 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:42.837632 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450}
W20260811 18:52:42.837917 11561 catalog_manager.cc:4584] TS e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7aa7822e5a11431e9b03e12bb01df7ad: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37477: connect: Connection refused (error 111)
I20260811 18:52:42.844350 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11958
I20260811 18:52:42.857211 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 11542
I20260811 18:52:42.867488 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--webserver_interface=127.27.115.254
--webserver_port=46569
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:42.992475 12156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:42.992794 12156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:42.992838 12156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:42.997102 12156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:42.997185 12156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:42.997205 12156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:42.997228 12156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:42.997244 12156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:43.003012 12156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=46569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:43.004361 12156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:43.005578 12156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:43.021629 12166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:43.021695 12164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.021829 12156 server_base.cc:1034] running on GCE node
W20260811 18:52:43.021631 12162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.022787 12156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:43.024259 12156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:43.025472 12156 hybrid_clock.cc:648] HybridClock initialized: now 1786474363025441 us; error 49 us; skew 500 ppm
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:43.029181 12156 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:43.030454 12156 webserver.cc:460] Webserver started at http://127.27.115.254:46569/ using document root <none> and password file <none>
I20260811 18:52:43.031157 12156 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:43.031296 12156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:43.036017 12156 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:52:43.038707 12174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.040129 12156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:43.040346 12156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "39042c5814fa4b07b7dd7bb710bde14a"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "f47c570e0484e689631450541a32cab2"
server_key_iv: "d0a7ecb7035eb1ccb01b4d7e3c4d4508"
server_key_version: "encryptionkey@0"
I20260811 18:52:43.041054 12156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:43.053040 12156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:43.056895 12156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:43.057183 12156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:43.065567 12156 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39939
I20260811 18:52:43.065933 12231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39939 every 8 connection(s)
I20260811 18:52:43.066958 12156 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:43.071944 12233 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:52:43.073937 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap starting.
I20260811 18:52:43.076421 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12156
I20260811 18:52:43.076948 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:37477
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=37273
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:43.099812 12233 log.cc:824] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:43.109724 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:52:43.110265 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap complete.
I20260811 18:52:43.114765 12233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:43.115185 12233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Initialized, Role: FOLLOWER
I20260811 18:52:43.115777 12233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:43.115937 12233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:43.116046 12233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:43.116163 12233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:52:43.118280 12233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:43.118654 12233 leader_election.cc:302] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39042c5814fa4b07b7dd7bb710bde14a; no voters:
I20260811 18:52:43.119025 12233 leader_election.cc:288] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:52:43.119167 12238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:52:43.119380 12238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 LEADER]: Becoming Leader. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Running, Role: LEADER
I20260811 18:52:43.119695 12238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:43.120395 12233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:43.120776 12240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39042c5814fa4b07b7dd7bb710bde14a. Latest consensus state: current_term: 2 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:43.120913 12240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:43.121189 12239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:43.121304 12239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:43.121378 12242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:43.126922 12242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:43.127712 12242 catalog_manager.cc:1259] Loaded cluster ID: 277a48617dac498ebea4d1ec6193f995
I20260811 18:52:43.127789 12242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:43.129318 12242 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 18:52:43.134275 12242 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 18:52:43.140604 12242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:43.141573 12242 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Loaded TSK: 0
I20260811 18:52:43.142935 12242 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:52:43.212905 12235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:43.213250 12235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:43.213346 12235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:43.217763 12235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:43.217866 12235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:43.217967 12235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:52:43.224061 12235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:37477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=37273
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:43.225371 12235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:43.226589 12235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:43.238237 12268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:43.238224 12266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.238445 12235 server_base.cc:1034] running on GCE node
W20260811 18:52:43.238227 12265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.239169 12235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:43.239951 12235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:43.241144 12235 hybrid_clock.cc:648] HybridClock initialized: now 1786474363241134 us; error 67 us; skew 500 ppm
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:43.244298 12235 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:52:43.245443 12235 webserver.cc:460] Webserver started at http://127.27.115.193:37273/ using document root <none> and password file <none>
I20260811 18:52:43.246110 12235 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:43.246280 12235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:43.250792 12235 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 18:52:43.253568 12276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.255136 12235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:43.255293 12235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "8b9fb725e466aa1b56af45ee372c9ff5"
server_key_iv: "d2f56144c5ab6d598c5a3ac92befbb2a"
server_key_version: "encryptionkey@0"
I20260811 18:52:43.255910 12235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:43.290953 12235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:43.294462 12235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:43.294703 12235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:43.295442 12235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:43.296766 12283 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:52:43.301750 12235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:52:43.301867 12235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 18:52:43.301973 12235 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:52:43.304111 12235 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:52:43.304181 12235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:52:43.304656 12283 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap starting.
I20260811 18:52:43.315958 12235 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37477
I20260811 18:52:43.316059 12402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37477 every 8 connection(s)
I20260811 18:52:43.317185 12235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:43.326606 12283 log.cc:824] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:43.327313 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12235
I20260811 18:52:43.327772 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:34497
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=33969
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:43.336905 12283 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:52:43.337528 12283 tablet_bootstrap.cc:492] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap complete.
I20260811 18:52:43.339243 12283 ts_tablet_manager.cc:1397] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.015s
I20260811 18:52:43.341235 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.319700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56231 (local address 127.27.115.254:39939)
0811 18:52:43.320141 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.320162 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.320573 (+ 411us) server_negotiation.cc:407] Connection header received
0811 18:52:43.322080 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.322111 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.322708 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.323190 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:43.324346 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.325833 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.328825 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.329660 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.329864 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.333679 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:43.333687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:43.333706 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:43.333748 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:43.336472 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.337367 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.337379 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.337387 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.337597 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.338153 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.338159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.338162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.338732 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:43.338892 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.339775 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.340146 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":148,"threads_started":1}
I20260811 18:52:43.341356 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.319591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:56231)
0811 18:52:43.320361 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.320421 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.321531 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.323211 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.323218 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.323651 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:43.324138 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.324154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.326037 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.326040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.328500 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.328522 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.329980 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.329986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.330263 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.331132 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:43.331159 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:43.333369 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:43.336675 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.336686 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.336708 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.337176 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.337734 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.337741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.337744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.337980 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.338896 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:43.338907 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:43.339331 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.339617 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.340090 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":171,"threads_started":1}
I20260811 18:52:43.342744 12283 raft_consensus.cc:348] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:43.343372 12283 raft_consensus.cc:729] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bd57379d2f4fd5ae6e295a62c0c2b3, State: Initialized, Role: FOLLOWER
I20260811 18:52:43.343986 12283 consensus_queue.cc:260] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } }
I20260811 18:52:43.345170 12403 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:43.345283 12283 ts_tablet_manager.cc:1428] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 18:52:43.345458 12403 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:43.346172 12403 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
I20260811 18:52:43.349239 12193 ts_manager.cc:194] Registered new tserver with Master: e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477)
I20260811 18:52:43.353480 12193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:56231
I20260811 18:52:43.357117 12403 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:43.362354 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.352850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.0.0.1:44568)
0811 18:52:43.353944 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.353993 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.354195 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.357086 (+ 2891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.357094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.357127 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:43.357300 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.357309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.358786 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.358789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.360478 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.360492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.361706 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.361710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.361821 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.361861 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.361980 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.362185 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":137,"threads_started":1}
I20260811 18:52:43.362349 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.355182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.27.115.193:37477)
0811 18:52:43.356456 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.356469 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.356500 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:52:43.356592 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.356598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.356834 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.357030 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:43.357453 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.358623 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.360632 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.361495 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.361661 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.361807 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.362038 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.362178 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1121,"spinlock_wait_cycles":8960,"thread_start_us":495,"threads_started":1}
I20260811 18:52:43.363893 12331 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa7822e5a11431e9b03e12bb01df7ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:42 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:44568
I20260811 18:52:43.365501 12412 tablet_replica.cc:330] stopping tablet replica
I20260811 18:52:43.365885 12412 raft_consensus.cc:2229] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:43.366250 12412 raft_consensus.cc:2258] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:43.367579 12412 ts_tablet_manager.cc:1905] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:43.372310 12412 ts_tablet_manager.cc:1918] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:43.372432 12412 log.cc:1196] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/7aa7822e5a11431e9b03e12bb01df7ad
I20260811 18:52:43.372747 12412 ts_tablet_manager.cc:1939] T 7aa7822e5a11431e9b03e12bb01df7ad P e6bd57379d2f4fd5ae6e295a62c0c2b3: Deleting consensus metadata
I20260811 18:52:43.374977 12179 catalog_manager.cc:4841] TS e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477): tablet 7aa7822e5a11431e9b03e12bb01df7ad (table test-table [id=f27578d425e34892ae251ed71b4cd6f3]) successfully deleted
W20260811 18:52:43.465595 12408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:43.465953 12408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:43.466038 12408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:43.470420 12408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:43.470531 12408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:43.470633 12408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:52:43.476362 12408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:34497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=33969
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:43.477741 12408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:43.478976 12408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:43.490203 12419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:43.490203 12421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.490296 12408 server_base.cc:1034] running on GCE node
W20260811 18:52:43.490203 12418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.491099 12408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:43.491859 12408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:43.493069 12408 hybrid_clock.cc:648] HybridClock initialized: now 1786474363493030 us; error 56 us; skew 500 ppm
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:43.496161 12408 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:52:43.497272 12408 webserver.cc:460] Webserver started at http://127.27.115.194:33969/ using document root <none> and password file <none>
I20260811 18:52:43.497921 12408 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:43.498041 12408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:43.502579 12408 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:43.505195 12429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.506891 12408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:43.507063 12408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b719659bc0694944a78c64f5587b3988"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "16da74d4efd3e918e7dde1f377f2ab7a"
server_key_iv: "a1eee4efa1a63044e77e5ad010f14bc1"
server_key_version: "encryptionkey@0"
I20260811 18:52:43.507731 12408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:43.523456 12408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:43.527042 12408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:43.527297 12408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:43.528083 12408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:43.529111 12408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:43.529165 12408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.529208 12408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:43.529268 12408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.539912 12408 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:34497
I20260811 18:52:43.539930 12556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:34497 every 8 connection(s)
I20260811 18:52:43.541066 12408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:43.548964 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12408
I20260811 18:52:43.549419 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:33873
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=33937
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:43.556613 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.543276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:34631 (local address 127.27.115.254:39939)
0811 18:52:43.543419 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.543423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.544306 (+ 883us) server_negotiation.cc:407] Connection header received
0811 18:52:43.545846 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.545850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.545944 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.546048 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:43.547337 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.547883 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.549074 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.549383 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.549493 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.552428 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:43.552431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:43.552433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:43.552460 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:43.554228 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.554822 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.554826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.554828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.554927 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.555312 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.555316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.555318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.555535 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:43.555653 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.556266 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.556431 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:52:43.557533 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.543632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:34631)
0811 18:52:43.544146 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.544197 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.545614 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.546269 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.546276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.546680 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:43.547171 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.547186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.548030 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.548034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.548929 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.548941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.549556 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.549561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.549769 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.550570 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:43.550586 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:43.552228 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:43.554366 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.554373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.554393 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.554709 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.555027 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.555030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.555032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.555210 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.555661 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:43.555671 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:43.556003 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.556222 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.556601 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":135,"threads_started":1}
I20260811 18:52:43.558921 12557 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:43.559212 12557 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:43.559774 12557 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
I20260811 18:52:43.561069 12193 ts_manager.cc:194] Registered new tserver with Master: b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497)
I20260811 18:52:43.562029 12193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:34631
W20260811 18:52:43.671693 12561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:43.672024 12561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:43.672073 12561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:43.676328 12561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:43.676410 12561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:43.676554 12561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:52:43.682191 12561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:33873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=33937
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:43.683369 12561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:43.684418 12561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:43.694473 12567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:43.694460 12566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.694593 12561 server_base.cc:1034] running on GCE node
W20260811 18:52:43.694455 12569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:43.695219 12561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:43.695950 12561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:43.697156 12561 hybrid_clock.cc:648] HybridClock initialized: now 1786474363697143 us; error 29 us; skew 500 ppm
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:43.700630 12561 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:52:43.701699 12561 webserver.cc:460] Webserver started at http://127.27.115.195:33937/ using document root <none> and password file <none>
I20260811 18:52:43.702423 12561 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:43.702551 12561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:43.706852 12561 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:43.709088 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.710573 12561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:52:43.710752 12561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "e328205ecddb4f8eb415d05469a37c5f"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "b69dea988b399eb6612bbc013da920c6"
server_key_iv: "7f445b35e56fd5b7df73378ea8514103"
server_key_version: "encryptionkey@0"
I20260811 18:52:43.711457 12561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:43.728523 12561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:43.732812 12561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:43.733065 12561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:43.733784 12561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:43.734859 12561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:43.734937 12561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.735015 12561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:43.735059 12561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:43.745488 12561 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33873
I20260811 18:52:43.745534 12707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33873 every 8 connection(s)
I20260811 18:52:43.746744 12561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:43.748056 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12561
I20260811 18:52:43.758366 12405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.748868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.27.115.254:39939)
0811 18:52:43.749035 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.749041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.749329 (+ 288us) server_negotiation.cc:407] Connection header received
0811 18:52:43.749523 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.749526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.749607 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.749700 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:43.750612 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.751137 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.752526 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.752942 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.753070 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.754734 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:43.754738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:43.754741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:43.754769 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:43.756354 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.756896 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.756903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.756909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.757024 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.757353 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.757356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.757357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.757530 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:43.757803 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.757979 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.758091 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 18:52:43 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474363, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:52:43.764976 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.748871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:48119 (local address 127.27.115.254:39939)
0811 18:52:43.749197 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.749202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.750627 (+ 1425us) server_negotiation.cc:407] Connection header received
0811 18:52:43.752526 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.752530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.752582 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.752695 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:43.754554 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.755343 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.756545 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.756866 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.756997 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.760676 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:43.760679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:43.760681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:43.760708 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:43.762554 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.763231 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.763235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.763236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.763330 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:43.763739 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:43.763747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:43.763749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:43.763953 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:43.764066 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.764632 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.764781 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":260,"thread_start_us":84,"threads_started":1}
I20260811 18:52:43.765919 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.749359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:48119)
0811 18:52:43.750472 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.750538 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.752150 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.752970 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.752977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.753546 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:43.754377 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.754404 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.755491 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.755494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.756391 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.756401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.757031 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.757034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.757322 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.757991 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:43.758013 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:43.760263 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:43.762729 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.762744 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.762760 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.763094 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.763432 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:43.763435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:43.763437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:43.763606 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:43.764068 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:43.764078 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:43.764369 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.764572 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.764947 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":110,"threads_started":1}
I20260811 18:52:43.767257 12708 heartbeater.cc:347] Connected to a master server at 127.27.115.254:39939
I20260811 18:52:43.767556 12708 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:43.768124 12708 heartbeater.cc:511] Master 127.27.115.254:39939 requested a full tablet report, sending...
I20260811 18:52:43.769318 12193 ts_manager.cc:194] Registered new tserver with Master: e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
I20260811 18:52:43.769910 12193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:48119
I20260811 18:52:43.772636 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:43.777695 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.773266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.27.115.254:39939)
0811 18:52:43.773402 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.773405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.773625 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:52:43.773757 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.773759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.773866 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.773952 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:43.774251 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.774866 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.775807 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.776039 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.776131 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.776593 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:43.777063 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:43.777179 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.777460 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.777539 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:52:43.778566 12193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:43.781945 12193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:43.791509 12331 tablet_service.cc:1467] Processing CreateTablet for tablet 5dde700a815e4279b4fd085f51348848 (DEFAULT_TABLE table=test-table [id=9d28b69943a14420b0c76eae679eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:43.792580 12331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dde700a815e4279b4fd085f51348848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:43.796125 12283 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap starting.
I20260811 18:52:43.797187 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.790904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33873 (local address 127.0.0.1:33496)
0811 18:52:43.791750 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.791772 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.791885 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.792875 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.792878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.792920 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:43.793078 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.793086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.794370 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.794377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.795347 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.795362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.796727 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.796731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.796823 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.796847 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.796918 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.797006 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":31,"spinlock_wait_cycles":12928,"thread_start_us":137,"threads_started":1}
I20260811 18:52:43.797232 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.791395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.27.115.195:33873)
0811 18:52:43.792365 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.792375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.792398 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:52:43.792469 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.792474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.792698 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.792898 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:43.793224 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.794215 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.795503 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.796565 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.796739 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.796906 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.796964 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.797069 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":105,"threads_started":1}
I20260811 18:52:43.797281 12283 tablet_bootstrap.cc:654] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:43.799463 12283 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: No bootstrap required, opened a new log
I20260811 18:52:43.799492 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.790290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:34497 (local address 127.0.0.1:53482)
0811 18:52:43.790456 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.790477 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.790578 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.792879 (+ 2301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.792883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.792923 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:43.793064 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.793072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.795370 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.795373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.796236 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.796248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.798960 (+ 2712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.798964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.799083 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.799113 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.799251 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.799346 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"spinlock_wait_cycles":9728}
I20260811 18:52:43.799595 12283 ts_tablet_manager.cc:1397] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:52:43.800135 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.790685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.27.115.194:34497)
0811 18:52:43.792292 (+ 1607us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.792307 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.792334 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:52:43.792422 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.792428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.792686 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.793234 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:43.793454 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.795195 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.796705 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.798761 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.798928 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.799104 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.799315 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.799426 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":372,"threads_started":1}
I20260811 18:52:43.800472 12283 raft_consensus.cc:348] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.800597 12283 raft_consensus.cc:374] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:43.800629 12283 raft_consensus.cc:729] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bd57379d2f4fd5ae6e295a62c0c2b3, State: Initialized, Role: FOLLOWER
I20260811 18:52:43.801066 12283 consensus_queue.cc:260] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.801846 12635 tablet_service.cc:1467] Processing CreateTablet for tablet 5dde700a815e4279b4fd085f51348848 (DEFAULT_TABLE table=test-table [id=9d28b69943a14420b0c76eae679eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:43.803365 12635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dde700a815e4279b4fd085f51348848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:43.803280 12283 ts_tablet_manager.cc:1428] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:52:43.804455 12485 tablet_service.cc:1467] Processing CreateTablet for tablet 5dde700a815e4279b4fd085f51348848 (DEFAULT_TABLE table=test-table [id=9d28b69943a14420b0c76eae679eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:43.805756 12485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dde700a815e4279b4fd085f51348848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:43.813156 12721 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Bootstrap starting.
I20260811 18:52:43.814642 12722 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Bootstrap starting.
I20260811 18:52:43.816308 12721 tablet_bootstrap.cc:654] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:43.817104 12721 log.cc:824] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:43.817282 12722 tablet_bootstrap.cc:654] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:43.818271 12722 log.cc:824] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Log is configured to *not* fsync() on all Append() calls
W20260811 18:52:43.818718 12404 tablet.cc:2369] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:43.819732 12721 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: No bootstrap required, opened a new log
I20260811 18:52:43.819998 12721 ts_tablet_manager.cc:1397] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 18:52:43.820600 12722 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: No bootstrap required, opened a new log
I20260811 18:52:43.820818 12722 ts_tablet_manager.cc:1397] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:52:43.823830 12721 raft_consensus.cc:348] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.824087 12721 raft_consensus.cc:374] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:43.824120 12721 raft_consensus.cc:729] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328205ecddb4f8eb415d05469a37c5f, State: Initialized, Role: FOLLOWER
I20260811 18:52:43.824265 12722 raft_consensus.cc:348] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.824640 12721 consensus_queue.cc:260] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.824757 12722 raft_consensus.cc:374] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:43.824860 12722 raft_consensus.cc:729] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b719659bc0694944a78c64f5587b3988, State: Initialized, Role: FOLLOWER
I20260811 18:52:43.825369 12722 consensus_queue.cc:260] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.825476 12708 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:43.826013 12721 ts_tablet_manager.cc:1428] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:52:43.826293 12557 heartbeater.cc:503] Master 127.27.115.254:39939 was elected leader, sending a full tablet report...
I20260811 18:52:43.826406 12722 ts_tablet_manager.cc:1428] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:52:43.891741 12726 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:43.892006 12726 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.893360 12726 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873), e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477)
I20260811 18:52:43.897706 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.893600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33873 (local address 127.27.115.194:56173)
0811 18:52:43.893926 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.893947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.894071 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.894414 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.894420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.894441 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:43.894586 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.894594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.895453 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.895456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.896405 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.896417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.897182 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.897189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.897323 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.897347 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.897430 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.897539 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":47}
I20260811 18:52:43.897701 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.893767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:56173 (local address 127.27.115.195:33873)
0811 18:52:43.893939 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.893947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.894101 (+ 154us) server_negotiation.cc:407] Connection header received
0811 18:52:43.894215 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.894218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.894285 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.894374 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:43.894702 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.895316 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.896553 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.897008 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.897112 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.897157 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.897466 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.897552 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:52:43.897763 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.893837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:45323 (local address 127.27.115.193:37477)
0811 18:52:43.894167 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.894172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.894365 (+ 193us) server_negotiation.cc:407] Connection header received
0811 18:52:43.894448 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.894450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.894497 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.894592 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:43.894937 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.895530 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.896670 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.897157 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.897289 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.897344 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.897539 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.897630 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":101,"threads_started":1}
I20260811 18:52:43.897753 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.893718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.27.115.194:45323)
0811 18:52:43.894220 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.894234 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:43.894359 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:43.894611 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:43.894613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:43.894628 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:43.894778 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.894785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.895669 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.895671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.896535 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:43.896547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.897288 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.897291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:43.897375 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.897394 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:52:43.897495 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 18:52:43.897569 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":17,"thread_start_us":138,"threads_started":1}
I20260811 18:52:43.898653 12655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dde700a815e4279b4fd085f51348848" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e328205ecddb4f8eb415d05469a37c5f" is_pre_election: true
I20260811 18:52:43.898963 12353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dde700a815e4279b4fd085f51348848" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" is_pre_election: true
I20260811 18:52:43.899091 12655 raft_consensus.cc:2454] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 0.
I20260811 18:52:43.899271 12353 raft_consensus.cc:2454] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 0.
I20260811 18:52:43.899776 12431 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f, e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters:
I20260811 18:52:43.900084 12726 raft_consensus.cc:2781] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:43.900177 12726 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:43.900223 12726 raft_consensus.cc:3037] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:43.901497 12726 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.901988 12726 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 election: Requested vote from peers e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873), e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477)
I20260811 18:52:43.902453 12655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dde700a815e4279b4fd085f51348848" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e328205ecddb4f8eb415d05469a37c5f"
I20260811 18:52:43.902498 12353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dde700a815e4279b4fd085f51348848" candidate_uuid: "b719659bc0694944a78c64f5587b3988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
I20260811 18:52:43.902580 12655 raft_consensus.cc:3037] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:43.902755 12353 raft_consensus.cc:3037] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:43.903914 12353 raft_consensus.cc:2454] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 1.
I20260811 18:52:43.903921 12655 raft_consensus.cc:2454] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b719659bc0694944a78c64f5587b3988 in term 1.
I20260811 18:52:43.904459 12433 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b719659bc0694944a78c64f5587b3988, e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters:
I20260811 18:52:43.904768 12726 raft_consensus.cc:2781] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:43.905143 12726 raft_consensus.cc:686] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 1 LEADER]: Becoming Leader. State: Replica: b719659bc0694944a78c64f5587b3988, State: Running, Role: LEADER
I20260811 18:52:43.905608 12726 consensus_queue.cc:237] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:43.909756 12193 catalog_manager.cc:5495] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 reported cstate change: term changed from 0 to 1, leader changed from <none> to b719659bc0694944a78c64f5587b3988 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "b719659bc0694944a78c64f5587b3988" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } health_report { overall_health: UNKNOWN } } }
I20260811 18:52:43.934804 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:43.931128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.27.115.194:34497)
0811 18:52:43.931281 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:43.931285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:43.931299 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:52:43.931384 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:43.931386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:43.931444 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:43.931542 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:43.931809 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.932318 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.933208 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:43.933445 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:43.933534 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:43.933878 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:43.934414 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:43.934512 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:43.934608 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 18:52:43.934677 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:52:43.941960 12655 raft_consensus.cc:1261] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Refusing update from remote peer b719659bc0694944a78c64f5587b3988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:43.941956 12353 raft_consensus.cc:1261] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Refusing update from remote peer b719659bc0694944a78c64f5587b3988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:43.942888 12726 consensus_queue.cc:1035] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:43.943166 12730 consensus_queue.cc:1035] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:43.961831 12193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:43.962025 12193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:43.965277 12193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Sending DeleteTablet for 3 replicas of tablet 5dde700a815e4279b4fd085f51348848
I20260811 18:52:43.966210 12331 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dde700a815e4279b4fd085f51348848 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:43 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:44568
I20260811 18:52:43.966315 12485 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dde700a815e4279b4fd085f51348848 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:43 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:53482
I20260811 18:52:43.966432 12635 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dde700a815e4279b4fd085f51348848 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:43 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:33496
I20260811 18:52:43.966903 12743 tablet_replica.cc:330] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: stopping tablet replica
I20260811 18:52:43.967320 12743 raft_consensus.cc:2229] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:52:43.967387 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 12235
I20260811 18:52:43.973001 12743 raft_consensus.cc:2258] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:43.973574 12744 tablet_replica.cc:330] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: stopping tablet replica
I20260811 18:52:43.974103 12744 raft_consensus.cc:2229] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:43.974596 12744 raft_consensus.cc:2258] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:43.974898 12743 ts_tablet_manager.cc:1905] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:43.976475 12744 ts_tablet_manager.cc:1905] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:43.978897 12743 ts_tablet_manager.cc:1918] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:43.979051 12743 log.cc:1196] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/5dde700a815e4279b4fd085f51348848
I20260811 18:52:43.979450 12743 ts_tablet_manager.cc:1939] T 5dde700a815e4279b4fd085f51348848 P b719659bc0694944a78c64f5587b3988: Deleting consensus metadata
I20260811 18:52:43.979961 12744 ts_tablet_manager.cc:1918] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:43.980142 12744 log.cc:1196] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/5dde700a815e4279b4fd085f51348848
I20260811 18:52:43.980577 12744 ts_tablet_manager.cc:1939] T 5dde700a815e4279b4fd085f51348848 P e328205ecddb4f8eb415d05469a37c5f: Deleting consensus metadata
I20260811 18:52:43.981241 12177 catalog_manager.cc:4841] TS b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): tablet 5dde700a815e4279b4fd085f51348848 (table test-table [id=9d28b69943a14420b0c76eae679eb25d]) successfully deleted
W20260811 18:52:43.981359 12179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:37477 (error 108)
I20260811 18:52:43.982098 12176 catalog_manager.cc:4841] TS e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): tablet 5dde700a815e4279b4fd085f51348848 (table test-table [id=9d28b69943a14420b0c76eae679eb25d]) successfully deleted
W20260811 18:52:43.982806 12410 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:43.982459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37477 (local address 127.0.0.1:44576)
0811 18:52:43.982616 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:43.982702 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37477: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 18:52:43.983016 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 12408
W20260811 18:52:43.982982 12179 catalog_manager.cc:4584] TS e6bd57379d2f4fd5ae6e295a62c0c2b3 (127.27.115.193:37477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5dde700a815e4279b4fd085f51348848: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37477: connect: Connection reset by peer (error 104)
I20260811 18:52:43.999343 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 12561
I20260811 18:52:44.009853 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 12156
I20260811 18:52:44.021211 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--webserver_interface=127.27.115.254
--webserver_port=46569
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:39939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:44.143641 12747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:44.143967 12747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:44.144011 12747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:44.148211 12747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:44.148294 12747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:44.148317 12747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:44.148335 12747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:44.148351 12747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:44.154425 12747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:39939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:39939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=46569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:44.155644 12747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:44.156685 12747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:44.165650 12756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.165674 12754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.165664 12753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:44.165956 12747 server_base.cc:1034] running on GCE node
I20260811 18:52:44.166711 12747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:44.167912 12747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:44.169109 12747 hybrid_clock.cc:648] HybridClock initialized: now 1786474364169086 us; error 45 us; skew 500 ppm
Aug 11 18:52:44 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:44.172263 12747 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:52:44.173281 12747 webserver.cc:460] Webserver started at http://127.27.115.254:46569/ using document root <none> and password file <none>
I20260811 18:52:44.173907 12747 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:44.174006 12747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:44.178251 12747 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:44.180608 12764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.182052 12747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:52:44.182250 12747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "39042c5814fa4b07b7dd7bb710bde14a"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "f47c570e0484e689631450541a32cab2"
server_key_iv: "d0a7ecb7035eb1ccb01b4d7e3c4d4508"
server_key_version: "encryptionkey@0"
I20260811 18:52:44.182868 12747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:44.206768 12747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:44.210594 12747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:44.210901 12747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:44.219355 12747 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:39939
I20260811 18:52:44.219383 12826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:39939 every 8 connection(s)
I20260811 18:52:44.220644 12747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:44.225832 12827 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:52:44.227936 12827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap starting.
I20260811 18:52:44.229552 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12747
I20260811 18:52:44.230304 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:37477
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=37273
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:44.247833 12827 log.cc:824] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:44.261296 12827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:52:44.261828 12827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: Bootstrap complete.
I20260811 18:52:44.267192 12827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:44.267738 12827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Initialized, Role: FOLLOWER
I20260811 18:52:44.268404 12827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:44.268594 12827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:44.268697 12827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:44.268854 12827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:52:44.271399 12827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:44.271857 12827 leader_election.cc:302] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39042c5814fa4b07b7dd7bb710bde14a; no voters:
I20260811 18:52:44.272291 12827 leader_election.cc:288] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:52:44.272410 12831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:52:44.272653 12831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [term 3 LEADER]: Becoming Leader. State: Replica: 39042c5814fa4b07b7dd7bb710bde14a, State: Running, Role: LEADER
I20260811 18:52:44.272992 12831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } }
I20260811 18:52:44.273756 12827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:44.274698 12832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:44.274735 12833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39042c5814fa4b07b7dd7bb710bde14a. Latest consensus state: current_term: 3 leader_uuid: "39042c5814fa4b07b7dd7bb710bde14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39042c5814fa4b07b7dd7bb710bde14a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 39939 } } }
I20260811 18:52:44.274847 12832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:44.274870 12833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:44.278915 12849 catalog_manager.cc:1259] Loaded cluster ID: 277a48617dac498ebea4d1ec6193f995
I20260811 18:52:44.278976 12849 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: loading cluster ID for follower catalog manager: success
W20260811 18:52:44.279960 12849 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 39042c5814fa4b07b7dd7bb710bde14a: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260811 18:52:44.280030 12849 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260811 18:52:44.280192 12842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:44.282512 12842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:44.282975 12842 catalog_manager.cc:1259] Loaded cluster ID: 277a48617dac498ebea4d1ec6193f995
I20260811 18:52:44.283053 12842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 18:52:44.283628 12842 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f35fee609ad google::LogMessage::Fail() at ??:0
@ 0x7f35fee6388a google::LogMessage::SendToLog() at ??:0
@ 0x7f35fee604de google::LogMessage::Flush() at ??:0
@ 0x7f35fee61a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f360979ffdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f36097a0d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f360979d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f36097e1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f36010b23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f36010b2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f36010b43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f36010a3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f36026e76db start_thread at ??:0
@ 0x7f35fd95e61f clone at ??:0
W20260811 18:52:44.382236 12829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:44.382599 12829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:44.382682 12829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:44.387290 12829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:44.387398 12829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:44.387496 12829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:52:44.393668 12829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:37477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=37273
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:44.395043 12829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:44.396188 12829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:44.409108 12875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.409019 12873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.408998 12872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:44.409933 12829 server_base.cc:1034] running on GCE node
I20260811 18:52:44.410560 12829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:44.411386 12829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:44.412626 12829 hybrid_clock.cc:648] HybridClock initialized: now 1786474364412564 us; error 66 us; skew 500 ppm
Aug 11 18:52:44 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:44.416024 12829 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:52:44.417172 12829 webserver.cc:460] Webserver started at http://127.27.115.193:37273/ using document root <none> and password file <none>
I20260811 18:52:44.417840 12829 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:44.417943 12829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:44.422919 12829 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:44.425776 12883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.427392 12829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:44.427584 12829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3"
format_stamp: "Formatted at 2026-08-11 18:52:41 on dist-test-slave-9jjr"
server_key: "8b9fb725e466aa1b56af45ee372c9ff5"
server_key_iv: "d2f56144c5ab6d598c5a3ac92befbb2a"
server_key_version: "encryptionkey@0"
I20260811 18:52:44.428261 12829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:44.446851 12829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:44.450690 12829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:44.450923 12829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:44.451696 12829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:44.453207 12891 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:52:44.458941 12829 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:52:44.459178 12829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 18:52:44.459293 12829 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:52:44.461658 12829 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:52:44.461756 12829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:52:44.462075 12891 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap starting.
I20260811 18:52:44.474277 12829 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37477
I20260811 18:52:44.474625 13011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37477 every 8 connection(s)
I20260811 18:52:44.475746 12829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:44.480947 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 12829
I20260811 18:52:44.481475 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:34497
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=33969
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:44.488543 12891 log.cc:824] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:44.500954 12891 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:52:44.501502 12891 tablet_bootstrap.cc:492] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Bootstrap complete.
I20260811 18:52:44.503402 12891 ts_tablet_manager.cc:1397] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent bootstrapping tablet: real 0.042s user 0.024s sys 0.014s
I20260811 18:52:44.507565 12891 raft_consensus.cc:348] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:44.508284 12891 raft_consensus.cc:729] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bd57379d2f4fd5ae6e295a62c0c2b3, State: Initialized, Role: FOLLOWER
I20260811 18:52:44.508934 12891 consensus_queue.cc:260] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:44.510268 12891 ts_tablet_manager.cc:1428] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
W20260811 18:52:44.534050 13014 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:44.478562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:42917)
0811 18:52:44.479366 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.479443 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:44.480698 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:44.533011 (+ 52313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":248,"threads_started":1}
W20260811 18:52:44.534735 13012 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:39939 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:39939: Client connection negotiation failed: client connection to 127.27.115.254:39939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:52:44.535674 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.535347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:43205)
0811 18:52:44.535504 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.535587 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 18:52:44.536540 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.536297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:43533)
0811 18:52:44.536421 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.536477 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:52:44.536787 13012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:52:44.617087 13015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:44.617445 13015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:44.617537 13015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:44.621856 13015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:44.621973 13015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:44.622076 13015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:52:44.628453 13015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:34497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=33969
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:44.629833 13015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:44.631284 13015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:52:44.642688 13015 server_base.cc:1034] running on GCE node
W20260811 18:52:44.642613 13023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.642613 13022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.642648 13026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:44.643484 13015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:44.644359 13015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:44.645607 13015 hybrid_clock.cc:648] HybridClock initialized: now 1786474364645590 us; error 33 us; skew 500 ppm
Aug 11 18:52:44 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:44.649030 13015 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:52:44.650484 13015 webserver.cc:460] Webserver started at http://127.27.115.194:33969/ using document root <none> and password file <none>
I20260811 18:52:44.651165 13015 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:44.651290 13015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:44.656144 13015 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:52:44.660300 13033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.662240 13015 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:52:44.662405 13015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b719659bc0694944a78c64f5587b3988"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "16da74d4efd3e918e7dde1f377f2ab7a"
server_key_iv: "a1eee4efa1a63044e77e5ad010f14bc1"
server_key_version: "encryptionkey@0"
I20260811 18:52:44.663293 13015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:44.693535 13015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:44.697052 13015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:44.697289 13015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:44.697968 13015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:44.698925 13015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:44.698975 13015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.699018 13015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:44.699074 13015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.708886 13015 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:34497
I20260811 18:52:44.708909 13164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:34497 every 8 connection(s)
I20260811 18:52:44.710052 13015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:44.712990 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13015
I20260811 18:52:44.713550 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:33873
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:39939
--webserver_port=33937
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:44.715022 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.712594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:47893)
0811 18:52:44.713418 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.714033 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":179,"threads_started":1}
W20260811 18:52:44.715693 13165 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:39939 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:39939: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
I20260811 18:52:44.716439 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.716185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:53733)
0811 18:52:44.716312 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.716371 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 18:52:44.717203 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.716971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:34261)
0811 18:52:44.717083 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.717138 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:52:44.717401 13165 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:52:44.840584 13169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:44.840969 13169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:44.841048 13169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:44.845484 13169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:44.845649 13169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:44.845763 13169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:52:44.852073 13169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:33873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=33937
--tserver_master_addrs=127.27.115.254:39939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:52:44.853541 13169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:44.854825 13169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:44.867785 13174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.867789 13175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:44.867786 13177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:44.868363 13169 server_base.cc:1034] running on GCE node
I20260811 18:52:44.868919 13169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:44.869706 13169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:44.870905 13169 hybrid_clock.cc:648] HybridClock initialized: now 1786474364870877 us; error 45 us; skew 500 ppm
Aug 11 18:52:44 dist-test-slave-9jjr krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:44.874318 13169 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:52:44.875519 13169 webserver.cc:460] Webserver started at http://127.27.115.195:33937/ using document root <none> and password file <none>
I20260811 18:52:44.876187 13169 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:44.876260 13169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:44.880851 13169 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:52:44.883584 13185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.885210 13169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:44.885442 13169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "e328205ecddb4f8eb415d05469a37c5f"
format_stamp: "Formatted at 2026-08-11 18:52:42 on dist-test-slave-9jjr"
server_key: "b69dea988b399eb6612bbc013da920c6"
server_key_iv: "7f445b35e56fd5b7df73378ea8514103"
server_key_version: "encryptionkey@0"
I20260811 18:52:44.886220 13169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:44.908119 13169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:44.911999 13169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:44.912245 13169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:44.912950 13169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:44.913990 13169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:44.914042 13169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:44.914106 13169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:44.914258 13169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 18:52:44.924885 13169 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33873
I20260811 18:52:44.924988 13314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33873 every 8 connection(s)
I20260811 18:52:44.926105 13169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:44.930734 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.928756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:41493)
0811 18:52:44.929377 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.929836 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":156,"threads_started":1}
W20260811 18:52:44.931332 13315 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:39939 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:39939: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
I20260811 18:52:44.932040 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.931785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:48255)
0811 18:52:44.931904 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.931958 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:52:44.932739 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:44.932502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:45971)
0811 18:52:44.932603 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:44.932671 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:52:44.932928 13315 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:52:44.933209 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13169
I20260811 18:52:45.538095 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:45.537532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:48247)
0811 18:52:45.537910 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:45.537980 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":99,"threads_started":1}
I20260811 18:52:45.718647 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:45.718096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:44711)
0811 18:52:45.718454 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:45.718531 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":85,"threads_started":1}
I20260811 18:52:45.934044 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:45.933554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:47489)
0811 18:52:45.933862 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:45.933927 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":132,"threads_started":1}
I20260811 18:52:46.193931 13324 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:46.194247 13324 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:46.195446 13324 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
Aug 11 18:52:46 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.194@KRBTEST.COM
Aug 11 18:52:46 dist-test-slave-9jjr krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474364, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.195@KRBTEST.COM
I20260811 18:52:46.219024 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.195917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:34133 (local address 127.27.115.194:34497)
0811 18:52:46.196426 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:46.196446 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:46.196480 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:52:46.197161 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:46.197167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:46.197536 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:46.197862 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:46.199313 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.200316 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.201814 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.202389 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.202577 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:46.212532 (+ 9955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:46.212540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:46.212560 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:46.212603 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:46.215108 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216137 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:46.216145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:46.216151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:46.216335 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216863 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:46.216867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:46.216868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:46.217478 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:46.217628 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:46.218504 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 18:52:46.218840 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":127,"threads_started":1}
I20260811 18:52:46.219009 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.195751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33873 (local address 127.27.115.193:46029)
0811 18:52:46.196096 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:46.196114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:46.196236 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:46.197940 (+ 1704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:46.197946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:46.198560 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:46.199108 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:46.199125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.200485 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.200490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:46.201681 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:46.201693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.202529 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.202532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:46.202771 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:46.209839 (+ 7068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:46.209865 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:46.212250 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:46.215306 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:46.215315 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:46.215333 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:46.215982 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:46.216451 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:46.216457 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:46.216738 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:46.217619 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:46.217627 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:46.218050 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 18:52:46.218320 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 18:52:46.218818 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":267,"spinlock_wait_cycles":132608,"thread_start_us":74,"threads_started":1}
I20260811 18:52:46.219048 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.195937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:46029 (local address 127.27.115.195:33873)
0811 18:52:46.196420 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:46.196444 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:46.196480 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:52:46.197169 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:46.197182 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:46.197535 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:46.197862 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:46.199323 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.200315 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.201814 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.202390 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.202577 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:46.212532 (+ 9955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:46.212540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:46.212560 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:46.212603 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:46.215108 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216137 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:46.216145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:46.216151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:46.216335 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216863 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:46.216867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:46.216869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:46.217478 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:46.217628 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:46.218504 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 18:52:46.218840 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":101,"threads_started":1}
I20260811 18:52:46.219192 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.195767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:34497 (local address 127.27.115.193:34133)
0811 18:52:46.196161 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:46.196176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:46.196294 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:46.197940 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:46.197946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:46.198559 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:46.199108 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:46.199126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.200759 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.200762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:46.201675 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:46.201687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:46.202535 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:46.202539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:46.202772 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:46.209839 (+ 7067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:46.209864 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:46.212047 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:46.215306 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:46.215315 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:46.215333 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:46.215982 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:46.216451 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:46.216454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:46.216457 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:46.216738 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:46.217619 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:46.217627 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:46.218050 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 18:52:46.218343 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 18:52:46.218755 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":237,"spinlock_wait_cycles":92672,"thread_start_us":78,"threads_started":1}
W20260811 18:52:46.222337 12886 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
W20260811 18:52:46.222752 12885 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
I20260811 18:52:46.222873 12885 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f
I20260811 18:52:46.223233 13324 raft_consensus.cc:2726] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:52:46.539559 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.539146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:38663)
0811 18:52:46.539383 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:46.539459 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 18:52:46.720477 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.719815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:35633)
0811 18:52:46.720286 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:46.720366 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":132,"threads_started":1}
I20260811 18:52:46.935405 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:46.934910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:49369)
0811 18:52:46.935231 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:46.935295 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":93,"threads_started":1}
I20260811 18:52:47.541074 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:47.540578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:36819)
0811 18:52:47.540891 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:47.540966 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260811 18:52:47.722018 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:47.721508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:38337)
0811 18:52:47.721853 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:47.721918 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":89,"threads_started":1}
I20260811 18:52:47.937022 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:47.936450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:48305)
0811 18:52:47.936790 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:47.936882 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260811 18:52:48.438824 13334 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:48.438973 13334 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:48.439556 13334 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
W20260811 18:52:48.440529 12886 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
W20260811 18:52:48.440678 12885 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
I20260811 18:52:48.440789 12885 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f
I20260811 18:52:48.440984 13334 raft_consensus.cc:2726] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:52:48.542639 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:48.542087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:49743)
0811 18:52:48.542451 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:48.542516 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":167,"threads_started":1}
I20260811 18:52:48.723326 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:48.722858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:41841)
0811 18:52:48.723159 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:48.723222 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
I20260811 18:52:48.938550 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:48.937981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:55335)
0811 18:52:48.938372 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:48.938443 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":90,"threads_started":1}
I20260811 18:52:49.544400 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:49.543833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:59619)
0811 18:52:49.544227 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:49.544294 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":88,"threads_started":1}
I20260811 18:52:49.724967 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:49.724402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:52471)
0811 18:52:49.724759 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:49.724826 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260811 18:52:49.939942 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:49.939468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:47441)
0811 18:52:49.939769 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:49.939843 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":111,"threads_started":1}
I20260811 18:52:50.545977 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:50.545457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:59067)
0811 18:52:50.545805 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:50.545873 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":114,"threads_started":1}
I20260811 18:52:50.726727 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:50.726235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:33881)
0811 18:52:50.726548 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:50.726618 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
I20260811 18:52:50.941733 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:50.941153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:35885)
0811 18:52:50.941546 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:50.941624 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260811 18:52:50.994414 13347 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:50.994580 13347 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:50.994985 13347 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
W20260811 18:52:50.995859 12885 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
W20260811 18:52:50.995957 12886 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
I20260811 18:52:50.996071 12886 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f
I20260811 18:52:50.996279 13347 raft_consensus.cc:2726] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:52:51.547392 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:51.546895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:42967)
0811 18:52:51.547196 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:51.547265 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":90,"threads_started":1}
I20260811 18:52:51.728248 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:51.727709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:36511)
0811 18:52:51.728074 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:51.728144 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":85,"threads_started":1}
I20260811 18:52:51.943535 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:51.942905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:45159)
0811 18:52:51.943342 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:51.943418 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":121,"threads_started":1}
I20260811 18:52:52.548960 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:52.548400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:51253)
0811 18:52:52.548771 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:52.548851 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260811 18:52:52.729856 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:52.729346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:40613)
0811 18:52:52.729692 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:52.729756 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":141,"threads_started":1}
I20260811 18:52:52.945274 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:52.944749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:54019)
0811 18:52:52.945088 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:52.945164 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":90,"threads_started":1}
I20260811 18:52:53.550623 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:53.550053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:58873)
0811 18:52:53.550450 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:53.550519 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":176,"threads_started":1}
I20260811 18:52:53.731542 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:53.730981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:50505)
0811 18:52:53.731355 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:53.731428 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":104,"threads_started":1}
I20260811 18:52:53.947088 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:53.946439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:50427)
0811 18:52:53.946869 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:53.946965 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":152,"threads_started":1}
I20260811 18:52:54.354203 13358 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:54.354369 13358 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:54.354817 13358 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
W20260811 18:52:54.355820 12885 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
W20260811 18:52:54.355942 12886 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
I20260811 18:52:54.356010 12886 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f
I20260811 18:52:54.356194 13358 raft_consensus.cc:2726] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:52:54.552084 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:54.551629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:33665)
0811 18:52:54.551921 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:54.551983 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260811 18:52:54.733172 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:54.732608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:52949)
0811 18:52:54.732961 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:54.733033 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260811 18:52:54.948534 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:54.948033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:34109)
0811 18:52:54.948355 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:54.948423 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":118,"threads_started":1}
I20260811 18:52:55.553617 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:55.553154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:40489)
0811 18:52:55.553450 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:55.553512 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":79,"threads_started":1}
I20260811 18:52:55.734774 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:55.734277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:57343)
0811 18:52:55.734590 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:55.734661 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":92,"threads_started":1}
I20260811 18:52:55.950105 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:55.949545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:52279)
0811 18:52:55.949907 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:55.949981 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":120,"threads_started":1}
I20260811 18:52:56.555335 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:56.554820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:51913)
0811 18:52:56.555155 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:56.555228 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":92,"threads_started":1}
I20260811 18:52:56.736307 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:56.735804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:44583)
0811 18:52:56.736121 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:56.736186 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260811 18:52:56.951754 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:56.951152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:51571)
0811 18:52:56.951496 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:56.951567 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260811 18:52:57.556818 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:57.556337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:32913)
0811 18:52:57.556649 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:57.556713 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260811 18:52:57.737874 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:57.737407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:55789)
0811 18:52:57.737704 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:57.737771 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260811 18:52:57.812323 13373 raft_consensus.cc:482] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:57.812546 13373 raft_consensus.cc:504] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b719659bc0694944a78c64f5587b3988" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 34497 } } peers { permanent_uuid: "e328205ecddb4f8eb415d05469a37c5f" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33873 } } peers { permanent_uuid: "e6bd57379d2f4fd5ae6e295a62c0c2b3" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37477 } }
I20260811 18:52:57.813138 13373 leader_election.cc:288] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497), e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873)
W20260811 18:52:57.814034 12886 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b719659bc0694944a78c64f5587b3988 (127.27.115.194:34497): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
W20260811 18:52:57.814169 12885 leader_election.cc:341] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e328205ecddb4f8eb415d05469a37c5f (127.27.115.195:33873): Not found: Tablet not found: 5dde700a815e4279b4fd085f51348848
I20260811 18:52:57.814262 12885 leader_election.cc:302] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6bd57379d2f4fd5ae6e295a62c0c2b3; no voters: b719659bc0694944a78c64f5587b3988, e328205ecddb4f8eb415d05469a37c5f
I20260811 18:52:57.814471 13373 raft_consensus.cc:2726] T 5dde700a815e4279b4fd085f51348848 P e6bd57379d2f4fd5ae6e295a62c0c2b3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:52:57.953162 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:57.952661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:34835)
0811 18:52:57.952980 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:57.953048 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
I20260811 18:52:58.558187 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:58.557684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:58691)
0811 18:52:58.557945 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:58.557999 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260811 18:52:58.739398 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:58.738903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:33989)
0811 18:52:58.739229 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:58.739294 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260811 18:52:58.954562 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:58.954039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:38347)
0811 18:52:58.954370 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:58.954443 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":121,"threads_started":1}
I20260811 18:52:59.559926 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:59.559406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.193:35021)
0811 18:52:59.559759 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:59.559825 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":107,"threads_started":1}
I20260811 18:52:59.741061 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:59.740517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.194:59755)
0811 18:52:59.740888 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:59.740960 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":135,"threads_started":1}
I20260811 18:52:59.941893 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 12829
I20260811 18:52:59.952317 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13015
I20260811 18:52:59.958624 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:59.957886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:39939 (local address 127.27.115.195:50953)
0811 18:52:59.958343 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:59.958456 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":84,"threads_started":1}
I20260811 18:52:59.964699 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13169
2026-08-11T18:52:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19775 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:53:00 dist-test-slave-9jjr krb5kdc[13384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:53:00 dist-test-slave-9jjr krb5kdc[13384](info): set up 2 sockets
Aug 11 18:53:00 dist-test-slave-9jjr krb5kdc[13384](info): commencing operation
krb5kdc: starting...
W20260811 18:53:01.115506 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:53:01 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:53:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:53:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:53:01.285573 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:01.408476 13397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:01.408854 13397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:01.408900 13397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:01.413515 13397 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:53:01.413602 13397 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:53:01.413619 13397 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:53:01.413635 13397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:53:01.413653 13397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:01.413672 13397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:01.413689 13397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:53:01.413704 13397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:53:01.420755 13397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:43801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:01.422011 13397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:01.423245 13397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:01.434567 13403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:01.434561 13404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:01.434598 13397 server_base.cc:1034] running on GCE node
W20260811 18:53:01.434863 13406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:01.435771 13397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:01.437279 13397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:01.438530 13397 hybrid_clock.cc:648] HybridClock initialized: now 1786474381438500 us; error 53 us; skew 500 ppm
Aug 11 18:53:01 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:01.442519 13397 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:53:01.443773 13397 webserver.cc:460] Webserver started at http://127.27.115.254:42721/ using document root <none> and password file <none>
I20260811 18:53:01.444478 13397 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:01.444588 13397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:01.444870 13397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:53:01.447518 13397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "00881dbd851048e3becb22f1defcdaa9"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "34c4b021299822a47edb50bf8a0003f9"
server_key_iv: "aa542464e2dc9686b4cb13fea18bcb0d"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.448186 13397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "00881dbd851048e3becb22f1defcdaa9"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "34c4b021299822a47edb50bf8a0003f9"
server_key_iv: "aa542464e2dc9686b4cb13fea18bcb0d"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.452951 13397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:53:01.456077 13414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:01.457569 13397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:53:01.457760 13397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "00881dbd851048e3becb22f1defcdaa9"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "34c4b021299822a47edb50bf8a0003f9"
server_key_iv: "aa542464e2dc9686b4cb13fea18bcb0d"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.457911 13397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:01.489596 13397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:01.493157 13397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:01.493391 13397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:01.501451 13397 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:43801
I20260811 18:53:01.501801 13473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:43801 every 8 connection(s)
I20260811 18:53:01.502777 13397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:53:01.505169 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13397
I20260811 18:53:01.505354 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:53:01.505720 28111 external_mini_cluster.cc:1428] Setting key 1eee9a0b03b2088e54f17a95a02a29d3
I20260811 18:53:01.506155 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:01.513371 13474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap starting.
Aug 11 18:53:01 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:01.516847 13474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:01.518105 13474 log.cc:824] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:01.521088 13474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: No bootstrap required, opened a new log
I20260811 18:53:01.523147 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:01.507648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.27.115.254:43801)
0811 18:53:01.508178 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:01.508197 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:01.508236 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:53:01.509323 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:01.509346 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:01.509714 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:01.510104 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:01.511036 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.512089 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.512988 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.513504 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.513699 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:01.516936 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:01.516944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:01.516964 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:01.517010 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:01.519451 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:01.520051 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:01.520058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:01.520064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:01.520221 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:01.520522 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:01.520525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:01.520527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:01.521083 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:01.521293 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:01.521706 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:53:01.522050 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":180,"threads_started":1}
I20260811 18:53:01.525889 13474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:01.526204 13474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:01.526257 13474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Initialized, Role: FOLLOWER
I20260811 18:53:01.526683 13474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:01.526814 13474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:53:01.526856 13474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:53:01.526947 13474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:01.528215 13474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:01.528594 13474 leader_election.cc:302] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00881dbd851048e3becb22f1defcdaa9; no voters:
I20260811 18:53:01.528944 13474 leader_election.cc:288] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:53:01.529170 13480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:53:01.529496 13480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 LEADER]: Becoming Leader. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Running, Role: LEADER
I20260811 18:53:01.529829 13480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:01.530447 13474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:53:01.531677 13483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00881dbd851048e3becb22f1defcdaa9. Latest consensus state: current_term: 1 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:01.531834 13483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:01.532089 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:01.532251 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:01.532276 13491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:53:01.536273 13491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:53:01.543665 13491 catalog_manager.cc:1347] Generated new cluster ID: 8568f4fa785b4340aec611e5f86acfb8
I20260811 18:53:01.543777 13491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:53:01.552042 13491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:53:01.553267 13491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:53:01.561771 13491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Generated new TSK 0
I20260811 18:53:01.562743 13491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:53:01.599357 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:01.734408 13505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:01.734791 13505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:01.734881 13505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:01.739477 13505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:01.739616 13505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:01.739727 13505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:53:01.745906 13505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:01.747287 13505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:01.748742 13505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:01.760934 13516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:01.760934 13511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:01.760934 13510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:01.761247 13505 server_base.cc:1034] running on GCE node
I20260811 18:53:01.761811 13505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:01.762655 13505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:01.763887 13505 hybrid_clock.cc:648] HybridClock initialized: now 1786474381763867 us; error 41 us; skew 500 ppm
Aug 11 18:53:01 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:01.767467 13505 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:53:01.768579 13505 webserver.cc:460] Webserver started at http://127.27.115.193:37949/ using document root <none> and password file <none>
I20260811 18:53:01.769224 13505 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:01.769331 13505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:01.769588 13505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:53:01.771919 13505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "c932ddc2a3724745ab8f4c97bc178940"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "f58dfa19c09ffaf6fbad664f10579a76"
server_key_iv: "801e420f9c4fb049218c08bb0224cd13"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.772575 13505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "c932ddc2a3724745ab8f4c97bc178940"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "f58dfa19c09ffaf6fbad664f10579a76"
server_key_iv: "801e420f9c4fb049218c08bb0224cd13"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.776702 13505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:53:01.779690 13525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:01.780977 13505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:53:01.781137 13505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "c932ddc2a3724745ab8f4c97bc178940"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "f58dfa19c09ffaf6fbad664f10579a76"
server_key_iv: "801e420f9c4fb049218c08bb0224cd13"
server_key_version: "encryptionkey@0"
I20260811 18:53:01.781279 13505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:01.801664 13505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:01.805223 13505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:01.805478 13505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:01.806198 13505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:01.807160 13505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:01.807237 13505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:01.807338 13505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:01.807390 13505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:01.818405 13505 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37933
I20260811 18:53:01.818539 13650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37933 every 8 connection(s)
I20260811 18:53:01.819638 13505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:53:01.830094 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13505
I20260811 18:53:01.830305 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:53:01.830687 28111 external_mini_cluster.cc:1428] Setting key dfa7d033eab5d0dcd1874c653a7db05c
Aug 11 18:53:01 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474381, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:01.839054 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:01.821939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:38089 (local address 127.27.115.254:43801)
0811 18:53:01.822191 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:01.822199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:01.823434 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 18:53:01.824835 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:01.824839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:01.824910 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:01.825013 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:01.826745 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.827382 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.829121 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.829437 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.829540 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:01.833623 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:01.833634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:01.833636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:01.833666 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:01.835473 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:01.836371 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:01.836378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:01.836380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:01.836491 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:01.837075 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:01.837079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:01.837082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:01.837386 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:01.837502 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:01.838657 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0811 18:53:01.838844 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 18:53:01.840086 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:01.822588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:38089)
0811 18:53:01.823236 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:01.823296 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:01.824547 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:01.825382 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:01.825391 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:01.825865 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:01.826530 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:01.826547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.827594 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.827597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:01.828920 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:01.828933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:01.829685 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:01.829690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:01.829890 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:01.830906 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:01.830933 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:01.833403 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:01.835654 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:01.835666 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:01.835692 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:01.836185 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:01.836656 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:01.836659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:01.836661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:01.836925 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:01.837609 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:01.837618 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:01.838249 (+ 631us) client_negotiation.cc:769] Sending connection context
0811 18:53:01.838529 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 18:53:01.838880 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":153,"threads_started":1}
I20260811 18:53:01.841691 13653 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:01.842020 13653 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:01.842832 13653 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
I20260811 18:53:01.845325 13435 ts_manager.cc:194] Registered new tserver with Master: c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933)
I20260811 18:53:01.847256 13435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:38089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:53:01.870723 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:01.996229 13658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:01.996541 13658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:01.996589 13658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:02.000794 13658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:02.000872 13658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:02.000933 13658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:53:02.006844 13658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:02.008013 13658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:02.009153 13658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:02.020588 13663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:02.020640 13666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:02.020640 13664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:02.020723 13658 server_base.cc:1034] running on GCE node
I20260811 18:53:02.021508 13658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:02.022428 13658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:02.023646 13658 hybrid_clock.cc:648] HybridClock initialized: now 1786474382023630 us; error 37 us; skew 500 ppm
Aug 11 18:53:02 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:02.027220 13658 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:53:02.028373 13658 webserver.cc:460] Webserver started at http://127.27.115.194:46613/ using document root <none> and password file <none>
I20260811 18:53:02.029052 13658 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:02.029167 13658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:02.029434 13658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:53:02.032207 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "b30a430e35154648aadf335ae1913c16"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "b8cbc5d1bfec76a9e1ac8fe04b234d3e"
server_key_iv: "12c0a73679317e988983d385482fb8aa"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.032899 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "b30a430e35154648aadf335ae1913c16"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "b8cbc5d1bfec76a9e1ac8fe04b234d3e"
server_key_iv: "12c0a73679317e988983d385482fb8aa"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.037395 13658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:53:02.040818 13675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.042449 13658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:02.042650 13658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b30a430e35154648aadf335ae1913c16"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "b8cbc5d1bfec76a9e1ac8fe04b234d3e"
server_key_iv: "12c0a73679317e988983d385482fb8aa"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.042819 13658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:02.056571 13658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:02.060662 13658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:02.060937 13658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:02.061939 13658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:02.063097 13658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:02.063150 13658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.063196 13658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:02.063212 13658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.073383 13658 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:43011
I20260811 18:53:02.073403 13798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:43011 every 8 connection(s)
I20260811 18:53:02.074533 13658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:53:02.079679 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13658
I20260811 18:53:02.079794 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:53:02.080080 28111 external_mini_cluster.cc:1428] Setting key 92e1effb95c65c83cb86a5ca61096714
Aug 11 18:53:02 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:02.092224 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.076621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:42323 (local address 127.27.115.254:43801)
0811 18:53:02.076783 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.076786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.077980 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 18:53:02.079059 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.079062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.079128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.079242 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:02.080834 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.081388 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.083309 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.083569 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.083688 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.087313 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:02.087321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:02.087327 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:02.087373 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:02.089521 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.090261 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.090267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.090269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.090377 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.090802 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.090808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.090813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.091102 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:02.091217 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.091853 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.092020 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:53:02.093065 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.077025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:42323)
0811 18:53:02.077799 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.077849 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.078828 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.079446 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.079454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.079936 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:02.080648 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.080676 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.081535 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.081545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.083107 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.083127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.083784 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.083791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.084066 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.085013 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:02.085033 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:02.087039 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:02.089719 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:02.089725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:02.089744 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:02.090077 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:02.090489 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:02.090491 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:02.090493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:02.090653 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:02.091233 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:02.091246 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:02.091534 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.091705 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.092023 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":129,"threads_started":1}
I20260811 18:53:02.094533 13799 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:02.094792 13799 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:02.095345 13799 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:53:02.096798 13435 ts_manager.cc:194] Registered new tserver with Master: b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:02.097689 13435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:42323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:53:02.115794 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:02.240051 13804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:02.240427 13804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:02.240505 13804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:02.245129 13804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:02.245246 13804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:02.245349 13804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:53:02.251529 13804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:02.252820 13804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:02.253919 13804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:02.264551 13810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:02.264545 13812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:02.264545 13809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:02.264755 13804 server_base.cc:1034] running on GCE node
I20260811 18:53:02.265312 13804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:02.266006 13804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:02.267194 13804 hybrid_clock.cc:648] HybridClock initialized: now 1786474382267150 us; error 58 us; skew 500 ppm
Aug 11 18:53:02 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:02.270378 13804 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:53:02.271503 13804 webserver.cc:460] Webserver started at http://127.27.115.195:42841/ using document root <none> and password file <none>
I20260811 18:53:02.272104 13804 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:02.272169 13804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:02.272353 13804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:53:02.274647 13804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "66fe4db7e0c4e99676315bfdf04fd105"
server_key_iv: "94323409c3163e67adb03dcf89976b7a"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.275218 13804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "66fe4db7e0c4e99676315bfdf04fd105"
server_key_iv: "94323409c3163e67adb03dcf89976b7a"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.279374 13804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:53:02.282116 13819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.283674 13804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:02.283888 13804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "66fe4db7e0c4e99676315bfdf04fd105"
server_key_iv: "94323409c3163e67adb03dcf89976b7a"
server_key_version: "encryptionkey@0"
I20260811 18:53:02.284039 13804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:02.320044 13804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:02.324373 13804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:02.324642 13804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:02.325440 13804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:02.326557 13804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:02.326623 13804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.326673 13804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:02.326689 13804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:02.337493 13804 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33179
I20260811 18:53:02.337520 13950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33179 every 8 connection(s)
I20260811 18:53:02.338618 13804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:53:02.345487 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 13804
I20260811 18:53:02.345670 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:53:02.345992 28111 external_mini_cluster.cc:1428] Setting key 4cd4679dcaeec3bc5c1b71d7da65fb2f
Aug 11 18:53:02 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:02.354380 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.340884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:49727 (local address 127.27.115.254:43801)
0811 18:53:02.341042 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.341046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.341968 (+ 922us) server_negotiation.cc:407] Connection header received
0811 18:53:02.343470 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.343474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.343535 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.343646 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:02.344985 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.345567 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.347069 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.347412 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.347531 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.350638 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:02.350642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:02.350644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:02.350676 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:02.352164 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.352743 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.352746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.352747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.352834 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.353193 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.353195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.353196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.353396 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:02.353473 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.354021 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.354232 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:53:02.355284 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.341286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:49727)
0811 18:53:02.341803 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.341853 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.343181 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.343840 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.343847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.344277 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:02.344812 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.344827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.345727 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.345731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.346919 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.346932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.347606 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.347617 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.347830 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.348690 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:02.348717 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:02.350436 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:02.352306 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:02.352312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:02.352329 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:02.352626 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:02.352920 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:02.352922 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:02.352924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:02.353074 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:02.353487 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:02.353493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:02.353767 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.353938 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.354339 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":126,"threads_started":1}
I20260811 18:53:02.356736 13951 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:02.356994 13951 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:02.357532 13951 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
I20260811 18:53:02.358814 13435 ts_manager.cc:194] Registered new tserver with Master: 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179)
I20260811 18:53:02.359505 13435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:49727
I20260811 18:53:02.360827 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:53:02.372056 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.363371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.27.115.254:43801)
0811 18:53:02.363521 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.363530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.363709 (+ 179us) server_negotiation.cc:407] Connection header received
0811 18:53:02.363818 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.363820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.363864 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.363950 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:02.364784 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.365347 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.366433 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.366673 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.366767 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.368463 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:02.368467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:02.368469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:02.368502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:02.370023 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.370677 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.370681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.370682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.370781 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:02.371048 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:02.371050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:02.371051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:02.371328 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:02.371447 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.371773 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.371905 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:53:02.375547 13435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:53:02.379223 13435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:53:02.396772 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.390609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.27.115.193:37933)
0811 18:53:02.391598 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.391605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.391626 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:53:02.391699 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.391705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.391871 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.392482 (+ 611us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:02.392638 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.393699 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.395096 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.395838 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.395984 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.396163 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.396418 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.396583 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":75,"threads_started":1}
I20260811 18:53:02.396766 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.389976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.0.0.1:33310)
0811 18:53:02.391086 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.391127 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.391299 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.391929 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.391938 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.391971 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:02.392224 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.392235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.393309 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.393313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.394976 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.394988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.396318 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.396324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.396421 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.396444 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.396513 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.396579 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":132,"spinlock_wait_cycles":2304,"thread_start_us":156,"threads_started":1}
I20260811 18:53:02.396981 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.389695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37933 (local address 127.0.0.1:40138)
0811 18:53:02.391062 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.391126 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.391298 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.392024 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.392028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.392049 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:02.392340 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.392348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.393835 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.393839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.394967 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.394977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.395988 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.395994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.396138 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.396195 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.396342 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.396836 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":115,"spinlock_wait_cycles":9088,"thread_start_us":103,"threads_started":1}
I20260811 18:53:02.397457 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.390234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.27.115.194:43011)
0811 18:53:02.390775 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.390791 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.391239 (+ 448us) server_negotiation.cc:407] Connection header received
0811 18:53:02.391475 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.391481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.391731 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.391904 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:02.392383 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.393141 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.395141 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.395969 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.396099 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.396242 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.397030 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.397324 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":111,"threads_started":1}
I20260811 18:53:02.397627 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.389720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33179 (local address 127.0.0.1:36150)
0811 18:53:02.391071 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.391127 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.391299 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.392024 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.392027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.392040 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:02.392340 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.392348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.394346 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.394350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.395465 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.395475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.397206 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.397209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.397322 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.397346 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.397406 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.397473 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"mutex_wait_us":117,"spinlock_wait_cycles":60416,"thread_start_us":130,"threads_started":1}
I20260811 18:53:02.398622 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.390620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.27.115.195:33179)
0811 18:53:02.391598 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.391606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.391625 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:53:02.391689 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.391693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.391872 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.392168 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:02.392487 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.394181 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.395618 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.396498 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.396687 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.398294 (+ 1607us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.398361 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.398477 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":160,"threads_started":1}
I20260811 18:53:02.400677 13580 tablet_service.cc:1467] Processing CreateTablet for tablet a4770862c2b241228114d0c925b18cde (DEFAULT_TABLE table=test-table [id=a03659c78dcc444095fe77821591af98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:02.400746 13728 tablet_service.cc:1467] Processing CreateTablet for tablet a4770862c2b241228114d0c925b18cde (DEFAULT_TABLE table=test-table [id=a03659c78dcc444095fe77821591af98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:02.401934 13728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4770862c2b241228114d0c925b18cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:02.401937 13580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4770862c2b241228114d0c925b18cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:02.401839 13877 tablet_service.cc:1467] Processing CreateTablet for tablet a4770862c2b241228114d0c925b18cde (DEFAULT_TABLE table=test-table [id=a03659c78dcc444095fe77821591af98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:02.403079 13877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4770862c2b241228114d0c925b18cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:02.409142 13970 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Bootstrap starting.
I20260811 18:53:02.410683 13971 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Bootstrap starting.
I20260811 18:53:02.410863 13972 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Bootstrap starting.
I20260811 18:53:02.411931 13970 tablet_bootstrap.cc:654] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:02.412930 13970 log.cc:824] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:02.413220 13971 tablet_bootstrap.cc:654] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:02.413571 13972 tablet_bootstrap.cc:654] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:02.414209 13971 log.cc:824] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:02.414558 13972 log.cc:824] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:02.415527 13970 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: No bootstrap required, opened a new log
I20260811 18:53:02.415791 13970 ts_tablet_manager.cc:1397] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:53:02.416150 13971 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: No bootstrap required, opened a new log
I20260811 18:53:02.416355 13971 ts_tablet_manager.cc:1397] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:53:02.419904 13971 raft_consensus.cc:348] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.420244 13971 raft_consensus.cc:374] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:02.420105 13970 raft_consensus.cc:348] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.420344 13971 raft_consensus.cc:729] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b30a430e35154648aadf335ae1913c16, State: Initialized, Role: FOLLOWER
I20260811 18:53:02.420466 13970 raft_consensus.cc:374] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:02.420554 13970 raft_consensus.cc:729] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c932ddc2a3724745ab8f4c97bc178940, State: Initialized, Role: FOLLOWER
I20260811 18:53:02.420838 13972 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: No bootstrap required, opened a new log
I20260811 18:53:02.420945 13971 consensus_queue.cc:260] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.421207 13972 ts_tablet_manager.cc:1397] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 18:53:02.421128 13970 consensus_queue.cc:260] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.422345 13799 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
I20260811 18:53:02.422452 13971 ts_tablet_manager.cc:1428] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:53:02.422461 13970 ts_tablet_manager.cc:1428] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 18:53:02.422853 13653 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
I20260811 18:53:02.426388 13972 raft_consensus.cc:348] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.426779 13972 raft_consensus.cc:374] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:02.426867 13972 raft_consensus.cc:729] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea3bc53fa1c45dc82bc2d961ad86073, State: Initialized, Role: FOLLOWER
I20260811 18:53:02.427537 13972 consensus_queue.cc:260] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.428566 13951 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
I20260811 18:53:02.429055 13972 ts_tablet_manager.cc:1428] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:53:02.534240 13981 raft_consensus.cc:482] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:02.534552 13981 raft_consensus.cc:504] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.535933 13981 leader_election.cc:288] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:02.540103 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.536173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37933 (local address 127.27.115.195:51627)
0811 18:53:02.536475 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.536491 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.536611 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.536923 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.536926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.536946 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:02.537085 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.537091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.537886 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.537891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.538823 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.538835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.539579 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.539583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.539740 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.539770 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.539864 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.539953 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":42}
I20260811 18:53:02.540117 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.536343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:51627 (local address 127.27.115.193:37933)
0811 18:53:02.536488 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.536494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.536605 (+ 111us) server_negotiation.cc:407] Connection header received
0811 18:53:02.536747 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.536749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.536797 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.536906 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:02.537214 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.537757 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.538977 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.539434 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.539580 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.539641 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.539887 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.539968 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:53:02.540175 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.536403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:40671 (local address 127.27.115.194:43011)
0811 18:53:02.536547 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.536552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.536771 (+ 219us) server_negotiation.cc:407] Connection header received
0811 18:53:02.536917 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.536922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.536984 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.537083 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:02.537387 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.537892 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.539032 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.539459 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.539601 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.539660 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.539896 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.539984 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:53:02.540310 13982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.536311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.27.115.195:40671)
0811 18:53:02.536678 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.536692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:02.536793 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:02.537093 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:02.537096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:02.537113 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:02.537253 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.537259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.538017 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.538022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.538886 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:02.538897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.539597 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.539601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:02.539742 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.539774 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:53:02.539864 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:53:02.540154 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":216,"thread_start_us":156,"threads_started":1}
I20260811 18:53:02.541025 13602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4770862c2b241228114d0c925b18cde" candidate_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c932ddc2a3724745ab8f4c97bc178940" is_pre_election: true
I20260811 18:53:02.541025 13752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4770862c2b241228114d0c925b18cde" candidate_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30a430e35154648aadf335ae1913c16" is_pre_election: true
I20260811 18:53:02.541405 13752 raft_consensus.cc:2454] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea3bc53fa1c45dc82bc2d961ad86073 in term 0.
I20260811 18:53:02.541405 13602 raft_consensus.cc:2454] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea3bc53fa1c45dc82bc2d961ad86073 in term 0.
I20260811 18:53:02.542032 13820 leader_election.cc:302] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16, c932ddc2a3724745ab8f4c97bc178940; no voters:
I20260811 18:53:02.542447 13981 raft_consensus.cc:2781] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:53:02.542562 13981 raft_consensus.cc:482] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:53:02.542649 13981 raft_consensus.cc:3037] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:02.544025 13981 raft_consensus.cc:504] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.544469 13981 leader_election.cc:288] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [CANDIDATE]: Term 1 election: Requested vote from peers c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:02.544919 13602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4770862c2b241228114d0c925b18cde" candidate_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c932ddc2a3724745ab8f4c97bc178940"
I20260811 18:53:02.544934 13752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4770862c2b241228114d0c925b18cde" candidate_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30a430e35154648aadf335ae1913c16"
I20260811 18:53:02.545085 13602 raft_consensus.cc:3037] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:02.545095 13752 raft_consensus.cc:3037] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:02.546294 13602 raft_consensus.cc:2454] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea3bc53fa1c45dc82bc2d961ad86073 in term 1.
I20260811 18:53:02.546298 13752 raft_consensus.cc:2454] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea3bc53fa1c45dc82bc2d961ad86073 in term 1.
I20260811 18:53:02.546713 13820 leader_election.cc:302] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16; no voters:
I20260811 18:53:02.546965 13981 raft_consensus.cc:2781] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:53:02.547241 13981 raft_consensus.cc:686] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea3bc53fa1c45dc82bc2d961ad86073, State: Running, Role: LEADER
I20260811 18:53:02.547657 13981 consensus_queue.cc:237] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:02.551112 13434 catalog_manager.cc:5495] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } health_report { overall_health: UNKNOWN } } }
W20260811 18:53:02.571448 13654 tablet.cc:2369] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:53:02.575860 13800 tablet.cc:2369] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:53:02.589766 13952 tablet.cc:2369] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:53:02.592443 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.588460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.27.115.195:33179)
0811 18:53:02.588619 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:02.588625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:02.588640 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:53:02.588766 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:02.588768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:02.588844 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:02.588965 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:53:02.589228 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.589746 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.590863 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:02.591078 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:02.591188 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:02.591545 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:53:02.592002 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:53:02.592138 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:02.592213 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:53:02.592281 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:53:02.598963 13752 raft_consensus.cc:1261] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Refusing update from remote peer 2ea3bc53fa1c45dc82bc2d961ad86073: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:53:02.598963 13602 raft_consensus.cc:1261] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Refusing update from remote peer 2ea3bc53fa1c45dc82bc2d961ad86073: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:53:02.599874 13981 consensus_queue.cc:1035] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:53:02.600128 13984 consensus_queue.cc:1035] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:53:02.607719 13988 mvcc.cc:205] Tried to move back new op lower bound from 7317399071116738560 to 7317399070918316032. Current Snapshot: MvccSnapshot[applied={T|T < 7317399071116738560}]
I20260811 18:53:02.609849 13989 mvcc.cc:205] Tried to move back new op lower bound from 7317399071116738560 to 7317399070918316032. Current Snapshot: MvccSnapshot[applied={T|T < 7317399071116738560}]
I20260811 18:53:02.618433 13434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:53:02.618674 13434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:53:02.621904 13434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Sending DeleteTablet for 3 replicas of tablet a4770862c2b241228114d0c925b18cde
I20260811 18:53:02.622995 13877 tablet_service.cc:1514] Processing DeleteTablet for tablet a4770862c2b241228114d0c925b18cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:36150
I20260811 18:53:02.622998 13728 tablet_service.cc:1514] Processing DeleteTablet for tablet a4770862c2b241228114d0c925b18cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:33310
I20260811 18:53:02.623028 13580 tablet_service.cc:1514] Processing DeleteTablet for tablet a4770862c2b241228114d0c925b18cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:40138
I20260811 18:53:02.623677 13997 tablet_replica.cc:330] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: stopping tablet replica
I20260811 18:53:02.624073 13997 raft_consensus.cc:2229] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:53:02.624130 13998 tablet_replica.cc:330] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: stopping tablet replica
I20260811 18:53:02.624233 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13505
I20260811 18:53:02.624502 13998 raft_consensus.cc:2229] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:53:02.624519 13997 raft_consensus.cc:2258] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:02.624810 13998 raft_consensus.cc:2258] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:02.625293 14000 tablet_replica.cc:330] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: stopping tablet replica
I20260811 18:53:02.625892 14000 raft_consensus.cc:2229] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:53:02.626497 14000 raft_consensus.cc:2258] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:02.629786 13998 ts_tablet_manager.cc:1905] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:53:02.633412 13998 ts_tablet_manager.cc:1918] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:53:02.633666 13998 log.cc:1196] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/a4770862c2b241228114d0c925b18cde
I20260811 18:53:02.634227 13998 ts_tablet_manager.cc:1939] T a4770862c2b241228114d0c925b18cde P b30a430e35154648aadf335ae1913c16: Deleting consensus metadata
W20260811 18:53:02.635283 13417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:37933 (error 108)
I20260811 18:53:02.635587 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13658
I20260811 18:53:02.636736 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.636359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37933 (local address 127.0.0.1:40140)
0811 18:53:02.636567 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.636634 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260811 18:53:02.636974 13417 catalog_manager.cc:4584] TS c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4770862c2b241228114d0c925b18cde: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37933: connect: Connection refused (error 111)
I20260811 18:53:02.639482 14000 ts_tablet_manager.cc:1905] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:53:02.643540 14000 ts_tablet_manager.cc:1918] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:53:02.643692 14000 log.cc:1196] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/a4770862c2b241228114d0c925b18cde
I20260811 18:53:02.644129 14000 ts_tablet_manager.cc:1939] T a4770862c2b241228114d0c925b18cde P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting consensus metadata
I20260811 18:53:02.645879 13418 catalog_manager.cc:4841] TS 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): tablet a4770862c2b241228114d0c925b18cde (table test-table [id=a03659c78dcc444095fe77821591af98]) successfully deleted
I20260811 18:53:02.647428 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13804
I20260811 18:53:02.647724 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:02.647302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.0.0.1:33324)
0811 18:53:02.647482 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:02.647589 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.194:43011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 18:53:02.657778 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 13397
I20260811 18:53:22.669257 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--webserver_interface=127.27.115.254
--webserver_port=42721
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:22.797048 14002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:22.797420 14002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:22.797513 14002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:22.801839 14002 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:53:22.801957 14002 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:53:22.801987 14002 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:53:22.802024 14002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:53:22.802096 14002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:22.802181 14002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:22.802222 14002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:53:22.802259 14002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:53:22.808053 14002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:43801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=42721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:22.809316 14002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:22.810526 14002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:22.827296 14008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:22.827272 14011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:22.827272 14009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:22.827361 14002 server_base.cc:1034] running on GCE node
I20260811 18:53:22.828241 14002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:22.829609 14002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:22.830811 14002 hybrid_clock.cc:648] HybridClock initialized: now 1786474402830792 us; error 47 us; skew 500 ppm
Aug 11 18:53:22 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:22.834244 14002 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:53:22.835367 14002 webserver.cc:460] Webserver started at http://127.27.115.254:42721/ using document root <none> and password file <none>
I20260811 18:53:22.836051 14002 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:22.836163 14002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:22.841554 14002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:53:22.844394 14018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:22.846113 14002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:53:22.846370 14002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "00881dbd851048e3becb22f1defcdaa9"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "34c4b021299822a47edb50bf8a0003f9"
server_key_iv: "aa542464e2dc9686b4cb13fea18bcb0d"
server_key_version: "encryptionkey@0"
I20260811 18:53:22.847047 14002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:22.860365 14002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:22.864352 14002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:22.864600 14002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:22.873775 14002 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:43801
I20260811 18:53:22.874231 14077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:43801 every 8 connection(s)
I20260811 18:53:22.875280 14002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:53:22.878852 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14002
I20260811 18:53:22.879390 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:37933
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=37949
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:53:22.880438 14078 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:53:22.882532 14078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap starting.
I20260811 18:53:22.909770 14078 log.cc:824] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:22.921986 14078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:53:22.922534 14078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap complete.
I20260811 18:53:22.928980 14078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:22.929551 14078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Initialized, Role: FOLLOWER
I20260811 18:53:22.930194 14078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:22.930366 14078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:53:22.930444 14078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:53:22.930557 14078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:53:22.933331 14078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:22.933728 14078 leader_election.cc:302] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00881dbd851048e3becb22f1defcdaa9; no voters:
I20260811 18:53:22.934149 14078 leader_election.cc:288] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:53:22.934397 14084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:53:22.934784 14084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 LEADER]: Becoming Leader. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Running, Role: LEADER
I20260811 18:53:22.935139 14084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:22.935542 14078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:53:22.936579 14085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:22.936697 14085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:22.937136 14086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00881dbd851048e3becb22f1defcdaa9. Latest consensus state: current_term: 2 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:22.937237 14086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:22.937527 14094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:53:22.942756 14094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:53:22.943501 14094 catalog_manager.cc:1259] Loaded cluster ID: 8568f4fa785b4340aec611e5f86acfb8
I20260811 18:53:22.943574 14094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:53:22.946235 14094 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:53:22.946998 14094 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Loaded TSK: 0
I20260811 18:53:22.964901 14094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Generated new TSK 1
I20260811 18:53:22.965210 14094 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:53:23.021883 14082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:23.022291 14082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:23.022388 14082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:23.026923 14082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:23.027027 14082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:23.027134 14082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:53:23.032917 14082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:37933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=37949
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:23.034035 14082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:23.035308 14082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:23.048349 14113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.048307 14110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.048357 14111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:23.049115 14082 server_base.cc:1034] running on GCE node
I20260811 18:53:23.049857 14082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:23.050835 14082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:23.052029 14082 hybrid_clock.cc:648] HybridClock initialized: now 1786474403051991 us; error 58 us; skew 500 ppm
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:23.055394 14082 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:53:23.056581 14082 webserver.cc:460] Webserver started at http://127.27.115.193:37949/ using document root <none> and password file <none>
I20260811 18:53:23.057221 14082 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:23.057350 14082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:23.061817 14082 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:23.064507 14121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.065977 14082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:53:23.066124 14082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "c932ddc2a3724745ab8f4c97bc178940"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "f58dfa19c09ffaf6fbad664f10579a76"
server_key_iv: "801e420f9c4fb049218c08bb0224cd13"
server_key_version: "encryptionkey@0"
I20260811 18:53:23.066804 14082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:23.095949 14082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:23.099817 14082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:23.100067 14082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:23.100795 14082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:23.102087 14129 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:53:23.106700 14082 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:53:23.106828 14082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 18:53:23.106935 14082 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:53:23.109081 14082 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:53:23.109174 14082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:53:23.109556 14129 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Bootstrap starting.
I20260811 18:53:23.122779 14082 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37933
I20260811 18:53:23.122884 14250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37933 every 8 connection(s)
I20260811 18:53:23.124251 14082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:53:23.129814 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14082
I20260811 18:53:23.130483 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:43011
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=46613
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:53:23.133647 14129 log.cc:824] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:23.144389 14129 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:53:23.144923 14129 tablet_bootstrap.cc:492] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Bootstrap complete.
I20260811 18:53:23.146551 14129 ts_tablet_manager.cc:1397] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Time spent bootstrapping tablet: real 0.037s user 0.019s sys 0.017s
I20260811 18:53:23.148666 14254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.127158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:56975 (local address 127.27.115.254:43801)
0811 18:53:23.127752 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.127780 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.128235 (+ 455us) server_negotiation.cc:407] Connection header received
0811 18:53:23.129804 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.129823 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.130503 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.131084 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.133059 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.134359 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.136584 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.137473 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.137671 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.141907 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.141915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.141932 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.141976 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.144465 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.145240 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.145249 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.145257 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.145428 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.145906 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.145913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.145918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.146512 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.146660 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.147457 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.147775 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":182,"threads_started":1}
I20260811 18:53:23.148695 14256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.127243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:56975)
0811 18:53:23.127958 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.128036 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.129166 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.131202 (+ 2036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.131218 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.131926 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:23.132776 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.132810 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.134596 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.134604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.136370 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.136392 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.137708 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.137715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.138010 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.139108 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:23.139142 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:23.141551 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:23.144645 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.144652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.144669 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.145054 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.145582 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.145586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.145588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.145789 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.146737 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:23.146744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:23.147087 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.147302 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.147692 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":188,"threads_started":1}
I20260811 18:53:23.149770 14129 raft_consensus.cc:348] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:23.150256 14129 raft_consensus.cc:729] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c932ddc2a3724745ab8f4c97bc178940, State: Initialized, Role: FOLLOWER
I20260811 18:53:23.151011 14129 consensus_queue.cc:260] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } } peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } }
I20260811 18:53:23.152222 14251 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:23.152287 14129 ts_tablet_manager.cc:1428] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 18:53:23.152504 14251 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:23.153257 14251 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
I20260811 18:53:23.156451 14038 ts_manager.cc:194] Registered new tserver with Master: c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933)
I20260811 18:53:23.160653 14038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:56975
I20260811 18:53:23.163977 14251 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.193@KRBTEST.COM
I20260811 18:53:23.170794 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.159708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37933 (local address 127.0.0.1:38862)
0811 18:53:23.160169 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.160220 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.160360 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.161030 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.161036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.161505 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:23.161777 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.161786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.162981 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.162987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.163647 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.163657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.164350 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.164354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.164499 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.165166 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:23.165179 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:23.166814 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:23.168805 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.168814 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.168820 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.169075 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.169399 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.169403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.169405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.169496 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.169996 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:23.170002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:23.170211 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.170376 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.170602 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
I20260811 18:53:23.170855 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.159816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.27.115.193:37933)
0811 18:53:23.160520 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.160532 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.160560 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:53:23.160637 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.160643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.160857 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.160997 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.161940 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.162864 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.163783 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.164190 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.164317 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.166963 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.166974 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.166987 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.167021 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.168675 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.169200 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.169206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.169208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.169300 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.169600 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.169604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.169605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.169872 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.170025 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.170463 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.170653 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":596,"thread_start_us":146,"threads_started":1}
I20260811 18:53:23.172377 14174 tablet_service.cc:1514] Processing DeleteTablet for tablet a4770862c2b241228114d0c925b18cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:38862
I20260811 18:53:23.173352 14263 tablet_replica.cc:330] stopping tablet replica
I20260811 18:53:23.173696 14263 raft_consensus.cc:2229] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:53:23.173979 14263 raft_consensus.cc:2258] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:23.175225 14263 ts_tablet_manager.cc:1905] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:53:23.180083 14263 ts_tablet_manager.cc:1918] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:53:23.180209 14263 log.cc:1196] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal/wals/a4770862c2b241228114d0c925b18cde
I20260811 18:53:23.180495 14263 ts_tablet_manager.cc:1939] T a4770862c2b241228114d0c925b18cde P c932ddc2a3724745ab8f4c97bc178940: Deleting consensus metadata
I20260811 18:53:23.182988 14021 catalog_manager.cc:4841] TS c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933): tablet a4770862c2b241228114d0c925b18cde (table test-table [id=a03659c78dcc444095fe77821591af98]) successfully deleted
W20260811 18:53:23.262060 14257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:23.262396 14257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:23.262445 14257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:23.266716 14257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:23.266789 14257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:23.266891 14257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:53:23.272467 14257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:43011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=46613
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:23.273661 14257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:23.274787 14257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:53:23.285151 14257 server_base.cc:1034] running on GCE node
W20260811 18:53:23.285092 14269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.285136 14270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.285092 14272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:23.285907 14257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:23.286710 14257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:23.287904 14257 hybrid_clock.cc:648] HybridClock initialized: now 1786474403287889 us; error 31 us; skew 500 ppm
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:23.291309 14257 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:53:23.292264 14257 webserver.cc:460] Webserver started at http://127.27.115.194:46613/ using document root <none> and password file <none>
I20260811 18:53:23.292841 14257 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:23.292941 14257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:23.296953 14257 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:53:23.299440 14280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.301307 14257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:23.301501 14257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b30a430e35154648aadf335ae1913c16"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "b8cbc5d1bfec76a9e1ac8fe04b234d3e"
server_key_iv: "12c0a73679317e988983d385482fb8aa"
server_key_version: "encryptionkey@0"
I20260811 18:53:23.302124 14257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:23.315080 14257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:23.318848 14257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:23.319093 14257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:23.319841 14257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:23.320940 14257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:23.321024 14257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.321096 14257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:23.321141 14257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.331465 14257 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:43011
I20260811 18:53:23.331485 14412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:43011 every 8 connection(s)
I20260811 18:53:23.332625 14257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:53:23.340951 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14257
I20260811 18:53:23.341629 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:33179
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=42841
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:23.350492 14254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.334822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:59495 (local address 127.27.115.254:43801)
0811 18:53:23.335012 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.335016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.335988 (+ 972us) server_negotiation.cc:407] Connection header received
0811 18:53:23.337501 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.337505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.337573 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.337691 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.339247 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.339834 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.341279 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.341724 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.341884 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.345399 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.345406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.345412 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.345456 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.347809 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.348488 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.348494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.348499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.348622 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.349012 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.349017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.349021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.349325 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.349449 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.350023 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.350235 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 18:53:23.351404 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.335235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:59495)
0811 18:53:23.335826 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.335873 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.337264 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.337927 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.337935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.338403 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:23.339030 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.339048 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.340019 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.340022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.341047 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.341059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.342067 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.342071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.342481 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.343272 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:23.343293 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:23.345136 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:23.347988 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.347995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.348013 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.348357 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.348737 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.348739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.348741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.348904 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.349454 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:23.349460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:23.349748 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.349931 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.350338 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":126,"threads_started":1}
I20260811 18:53:23.352825 14413 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:23.353088 14413 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:23.353650 14413 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
I20260811 18:53:23.354897 14038 ts_manager.cc:194] Registered new tserver with Master: b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:23.355511 14038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:59495
W20260811 18:53:23.466351 14416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:23.466655 14416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:23.466702 14416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:23.471050 14416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:23.471165 14416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:23.471271 14416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:53:23.476998 14416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:33179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=42841
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:23.478340 14416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:23.479507 14416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:23.490511 14423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.490511 14424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:23.490543 14426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:23.490825 14416 server_base.cc:1034] running on GCE node
I20260811 18:53:23.491350 14416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:23.492107 14416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:23.493398 14416 hybrid_clock.cc:648] HybridClock initialized: now 1786474403493367 us; error 58 us; skew 500 ppm
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:23.496562 14416 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:53:23.497545 14416 webserver.cc:460] Webserver started at http://127.27.115.195:42841/ using document root <none> and password file <none>
I20260811 18:53:23.498207 14416 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:23.498324 14416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:23.502516 14416 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:23.504798 14435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.506353 14416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:23.506500 14416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "66fe4db7e0c4e99676315bfdf04fd105"
server_key_iv: "94323409c3163e67adb03dcf89976b7a"
server_key_version: "encryptionkey@0"
I20260811 18:53:23.507155 14416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:23.521636 14416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:23.525542 14416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:23.525835 14416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:23.526860 14416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:23.528131 14416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:23.528198 14416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.528262 14416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:23.528297 14416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:23.539773 14416 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33179
I20260811 18:53:23.539795 14564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33179 every 8 connection(s)
I20260811 18:53:23.541015 14416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:53:23.541747 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14416
I20260811 18:53:23.551616 14254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.542652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.27.115.254:43801)
0811 18:53:23.542806 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.542811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.543024 (+ 213us) server_negotiation.cc:407] Connection header received
0811 18:53:23.543202 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.543205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.543288 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.543388 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.544293 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.545056 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.546336 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.546550 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.546666 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.548108 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.548113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.548116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.548148 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.549643 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.550105 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.550111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.550119 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.550277 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.550610 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.550614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.550616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.550873 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.551108 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.551245 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.551389 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:53:23 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:53:23.556463 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.543685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:53567 (local address 127.27.115.254:43801)
0811 18:53:23.543981 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.543986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.544897 (+ 911us) server_negotiation.cc:407] Connection header received
0811 18:53:23.546532 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.546535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.546588 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.546698 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.547865 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.548367 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.549502 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.549741 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.549856 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.552799 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.552803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.552805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.552830 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.554415 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.554897 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.554899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.554900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.554977 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.555394 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.555396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.555397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.555569 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.555646 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.556222 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.556331 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":107,"threads_started":1}
I20260811 18:53:23.557433 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.544207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:53567)
0811 18:53:23.544759 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.544802 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.546315 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.546870 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.546877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.547237 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:23.547717 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.547731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.548494 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.548497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.549369 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.549378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.549903 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.549907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.550081 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.550766 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:23.550786 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:23.552625 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:23.554522 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.554528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.554542 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.554807 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.555131 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:23.555134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:23.555135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:23.555303 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:23.555672 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:23.555678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:23.555943 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.556149 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.556426 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":131,"threads_started":1}
I20260811 18:53:23.558934 14568 heartbeater.cc:347] Connected to a master server at 127.27.115.254:43801
I20260811 18:53:23.559221 14568 heartbeater.cc:464] Registering TS with master...
I20260811 18:53:23.559756 14568 heartbeater.cc:511] Master 127.27.115.254:43801 requested a full tablet report, sending...
I20260811 18:53:23.561057 14038 ts_manager.cc:194] Registered new tserver with Master: 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179)
I20260811 18:53:23.561666 14038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:53567
I20260811 18:53:23.565744 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:53:23.569963 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.566468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.27.115.254:43801)
0811 18:53:23.566597 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.566600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.566862 (+ 262us) server_negotiation.cc:407] Connection header received
0811 18:53:23.566951 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.566953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.567059 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.567157 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:53:23.567395 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.567907 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.568839 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.569067 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.569185 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.569480 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:53:23.569630 (+ 150us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 18:53:23.569855 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41284: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 18:53:23.570230 14571 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41284: token expired
W20260811 18:53:23.570542 13957 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.115.254:43801)
I20260811 18:53:23.578786 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.570974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.27.115.254:43801)
0811 18:53:23.571107 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.571111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.571124 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:53:23.571277 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.571281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.571326 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.571422 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:23.572112 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.572612 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.573440 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.573677 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.573789 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.575296 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:23.575300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:23.575303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:23.575343 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:23.576919 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.577439 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.577442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.577444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.577537 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:23.577831 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:23.577834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:23.577836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:23.578043 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:23.578221 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.578470 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.578612 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:53:23.581554 14038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:53:23.585137 14038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:53:23.594316 14174 tablet_service.cc:1467] Processing CreateTablet for tablet 75294834aaa74887b60cb9583513c07c (DEFAULT_TABLE table=test-table [id=6f54df00eefc482d9add016a318e19b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:23.595079 14174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75294834aaa74887b60cb9583513c07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:23.598029 14129 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Bootstrap starting.
I20260811 18:53:23.598631 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.593196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33179 (local address 127.0.0.1:46898)
0811 18:53:23.593347 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.593363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.593467 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.594447 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.594450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.594468 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:23.594590 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.594598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.595979 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.595982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.596958 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.596969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.598056 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.598060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.598291 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.598346 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.598401 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.598480 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1}
I20260811 18:53:23.599200 14129 tablet_bootstrap.cc:654] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:23.599387 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.593420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46898 (local address 127.27.115.195:33179)
0811 18:53:23.593896 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.593910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.593935 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:53:23.594027 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.594033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.594293 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.594468 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:23.594728 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.595854 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.597103 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.597917 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.598931 (+ 1014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.599097 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.599150 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.599242 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":207,"threads_started":1}
I20260811 18:53:23.600239 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.593457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.0.0.1:34726)
0811 18:53:23.593952 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.593966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.594052 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.595659 (+ 1607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.595665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.595691 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:23.595816 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.595823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.597337 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.597341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.598640 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.598655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.599755 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.599759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.599876 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.599911 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.599992 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.600085 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":47,"thread_start_us":144,"threads_started":1}
I20260811 18:53:23.600749 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.594746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.27.115.194:43011)
0811 18:53:23.595217 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.595227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.595249 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:53:23.595319 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.595324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.595500 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.595656 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:23.595965 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.597181 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.598791 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.599630 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.599759 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.599874 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.600497 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.600608 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":178,"threads_started":1}
I20260811 18:53:23.600869 14129 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: No bootstrap required, opened a new log
I20260811 18:53:23.600956 14129 ts_tablet_manager.cc:1397] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:53:23.601456 14129 raft_consensus.cc:348] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.601608 14129 raft_consensus.cc:374] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:23.601652 14129 raft_consensus.cc:729] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c932ddc2a3724745ab8f4c97bc178940, State: Initialized, Role: FOLLOWER
I20260811 18:53:23.601794 14129 consensus_queue.cc:260] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.602057 14129 ts_tablet_manager.cc:1428] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 18:53:23.602638 14489 tablet_service.cc:1467] Processing CreateTablet for tablet 75294834aaa74887b60cb9583513c07c (DEFAULT_TABLE table=test-table [id=6f54df00eefc482d9add016a318e19b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:23.603883 14489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75294834aaa74887b60cb9583513c07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:23.603865 14338 tablet_service.cc:1467] Processing CreateTablet for tablet 75294834aaa74887b60cb9583513c07c (DEFAULT_TABLE table=test-table [id=6f54df00eefc482d9add016a318e19b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:53:23.604941 14338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75294834aaa74887b60cb9583513c07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:53:23.611637 14582 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Bootstrap starting.
I20260811 18:53:23.611649 14583 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Bootstrap starting.
I20260811 18:53:23.614513 14583 tablet_bootstrap.cc:654] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:23.614514 14582 tablet_bootstrap.cc:654] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Neither blocks nor log segments found. Creating new log.
I20260811 18:53:23.615272 14583 log.cc:824] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:23.615272 14582 log.cc:824] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:23.617559 14582 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: No bootstrap required, opened a new log
I20260811 18:53:23.617559 14583 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: No bootstrap required, opened a new log
I20260811 18:53:23.617782 14583 ts_tablet_manager.cc:1397] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 18:53:23.618036 14582 ts_tablet_manager.cc:1397] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 18:53:23.622097 14583 raft_consensus.cc:348] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.622097 14582 raft_consensus.cc:348] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.622486 14582 raft_consensus.cc:374] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:23.622488 14583 raft_consensus.cc:374] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:53:23.622542 14582 raft_consensus.cc:729] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea3bc53fa1c45dc82bc2d961ad86073, State: Initialized, Role: FOLLOWER
I20260811 18:53:23.622553 14583 raft_consensus.cc:729] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b30a430e35154648aadf335ae1913c16, State: Initialized, Role: FOLLOWER
I20260811 18:53:23.623118 14583 consensus_queue.cc:260] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.623119 14582 consensus_queue.cc:260] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.624063 14568 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
I20260811 18:53:23.624307 14583 ts_tablet_manager.cc:1428] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 18:53:23.624332 14413 heartbeater.cc:503] Master 127.27.115.254:43801 was elected leader, sending a full tablet report...
I20260811 18:53:23.625211 14582 ts_tablet_manager.cc:1428] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
W20260811 18:53:23.625998 14252 tablet.cc:2369] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:53:23.639295 14259 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:23.639567 14259 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.640837 14259 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:23.645421 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.641337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:53997 (local address 127.27.115.194:43011)
0811 18:53:23.641473 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.641477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.641493 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:53:23.641545 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.641548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.641591 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.641665 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:23.641993 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.642783 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.644129 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.644782 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.644918 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.644975 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.645208 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.645285 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:53:23.645468 14256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.641154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.27.115.193:53997)
0811 18:53:23.641297 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.641315 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.641443 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.641705 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.641709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.641730 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:23.641867 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.641873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.642899 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.642902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.643998 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.644008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.644914 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.644918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.645042 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.645102 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.645189 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.645289 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 18:53:23.646086 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.641172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33179 (local address 127.27.115.193:58275)
0811 18:53:23.641893 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.641905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:23.642024 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:23.642397 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:23.642401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:23.642421 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:53:23.642544 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.642550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.643512 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.643516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.644494 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:23.644505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.645475 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.645636 (+ 161us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:23.645771 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.645796 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:53:23.645879 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:53:23.645960 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":354,"thread_start_us":136,"threads_started":1}
I20260811 18:53:23.646133 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.641240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:58275 (local address 127.27.115.195:33179)
0811 18:53:23.641377 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.641382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.642044 (+ 662us) server_negotiation.cc:407] Connection header received
0811 18:53:23.642230 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.642234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.642292 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.642365 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:53:23.642665 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.643385 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.644638 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.645232 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.645341 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.645392 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.645910 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.646020 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:53:23.646332 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75294834aaa74887b60cb9583513c07c" candidate_uuid: "c932ddc2a3724745ab8f4c97bc178940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30a430e35154648aadf335ae1913c16" is_pre_election: true
I20260811 18:53:23.646723 14360 raft_consensus.cc:2454] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c932ddc2a3724745ab8f4c97bc178940 in term 0.
I20260811 18:53:23.646953 14510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75294834aaa74887b60cb9583513c07c" candidate_uuid: "c932ddc2a3724745ab8f4c97bc178940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" is_pre_election: true
I20260811 18:53:23.647266 14510 raft_consensus.cc:2454] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c932ddc2a3724745ab8f4c97bc178940 in term 0.
I20260811 18:53:23.647316 14122 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b30a430e35154648aadf335ae1913c16, c932ddc2a3724745ab8f4c97bc178940; no voters:
I20260811 18:53:23.647810 14259 raft_consensus.cc:2781] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:53:23.647926 14259 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:53:23.647976 14259 raft_consensus.cc:3037] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:23.649111 14259 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.649722 14259 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 1 election: Requested vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
I20260811 18:53:23.650074 14510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75294834aaa74887b60cb9583513c07c" candidate_uuid: "c932ddc2a3724745ab8f4c97bc178940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
I20260811 18:53:23.650238 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75294834aaa74887b60cb9583513c07c" candidate_uuid: "c932ddc2a3724745ab8f4c97bc178940" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30a430e35154648aadf335ae1913c16"
I20260811 18:53:23.650318 14510 raft_consensus.cc:3037] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:23.650444 14360 raft_consensus.cc:3037] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:53:23.651564 14360 raft_consensus.cc:2454] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c932ddc2a3724745ab8f4c97bc178940 in term 1.
I20260811 18:53:23.651553 14510 raft_consensus.cc:2454] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c932ddc2a3724745ab8f4c97bc178940 in term 1.
I20260811 18:53:23.651898 14124 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea3bc53fa1c45dc82bc2d961ad86073, c932ddc2a3724745ab8f4c97bc178940; no voters:
I20260811 18:53:23.652103 14259 raft_consensus.cc:2781] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:53:23.652335 14259 raft_consensus.cc:686] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 LEADER]: Becoming Leader. State: Replica: c932ddc2a3724745ab8f4c97bc178940, State: Running, Role: LEADER
I20260811 18:53:23.652642 14259 consensus_queue.cc:237] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:23.655810 14038 catalog_manager.cc:5495] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 reported cstate change: term changed from 0 to 1, leader changed from <none> to c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193). New cstate: current_term: 1 leader_uuid: "c932ddc2a3724745ab8f4c97bc178940" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } health_report { overall_health: HEALTHY } } }
I20260811 18:53:23.684085 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.680047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.27.115.193:37933)
0811 18:53:23.680348 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:23.680352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:23.680363 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:53:23.680450 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:23.680452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:23.680507 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:23.680641 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:53:23.680898 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.681499 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.682457 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:23.682690 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:23.682838 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:23.683164 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:53:23.683617 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:53:23.683749 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:23.683839 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 18:53:23.683905 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260811 18:53:23.688512 14510 raft_consensus.cc:1261] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Refusing update from remote peer c932ddc2a3724745ab8f4c97bc178940: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:53:23.688612 14360 raft_consensus.cc:1261] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Refusing update from remote peer c932ddc2a3724745ab8f4c97bc178940: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:53:23.689302 14259 consensus_queue.cc:1035] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:53:23.689524 14592 consensus_queue.cc:1035] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:53:23.698489 14597 mvcc.cc:205] Tried to move back new op lower bound from 7317399157500608512 to 7317399157364256768. Current Snapshot: MvccSnapshot[applied={T|T < 7317399157500608512}]
I20260811 18:53:23.709910 14037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:53:23.710178 14037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:53:23.713248 14037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Sending DeleteTablet for 3 replicas of tablet 75294834aaa74887b60cb9583513c07c
I20260811 18:53:23.714160 14174 tablet_service.cc:1514] Processing DeleteTablet for tablet 75294834aaa74887b60cb9583513c07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:23 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:38862
I20260811 18:53:23.714185 14338 tablet_service.cc:1514] Processing DeleteTablet for tablet 75294834aaa74887b60cb9583513c07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:23 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:34726
I20260811 18:53:23.714342 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14082
I20260811 18:53:23.716899 14608 tablet_replica.cc:330] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: stopping tablet replica
I20260811 18:53:23.717499 14489 tablet_service.cc:1514] Processing DeleteTablet for tablet 75294834aaa74887b60cb9583513c07c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:53:23 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46898
I20260811 18:53:23.717347 14608 raft_consensus.cc:2229] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:53:23.718078 14609 tablet_replica.cc:330] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: stopping tablet replica
I20260811 18:53:23.718116 14608 raft_consensus.cc:2258] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:23.718482 14609 raft_consensus.cc:2229] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:53:23.718861 14609 raft_consensus.cc:2258] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:53:23.720520 14608 ts_tablet_manager.cc:1905] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:53:23.720734 14609 ts_tablet_manager.cc:1905] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:53:23.723165 14608 ts_tablet_manager.cc:1918] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:53:23.723307 14608 log.cc:1196] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/75294834aaa74887b60cb9583513c07c
I20260811 18:53:23.723644 14608 ts_tablet_manager.cc:1939] T 75294834aaa74887b60cb9583513c07c P b30a430e35154648aadf335ae1913c16: Deleting consensus metadata
I20260811 18:53:23.724977 14609 ts_tablet_manager.cc:1918] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:53:23.724998 14020 catalog_manager.cc:4841] TS b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): tablet 75294834aaa74887b60cb9583513c07c (table test-table [id=6f54df00eefc482d9add016a318e19b0]) successfully deleted
I20260811 18:53:23.725102 14609 log.cc:1196] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/75294834aaa74887b60cb9583513c07c
I20260811 18:53:23.725648 14609 ts_tablet_manager.cc:1939] T 75294834aaa74887b60cb9583513c07c P 2ea3bc53fa1c45dc82bc2d961ad86073: Deleting consensus metadata
I20260811 18:53:23.727463 14022 catalog_manager.cc:4841] TS 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): tablet 75294834aaa74887b60cb9583513c07c (table test-table [id=6f54df00eefc482d9add016a318e19b0]) successfully deleted
W20260811 18:53:23.727917 14021 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:37933 (error 108)
I20260811 18:53:23.729031 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14257
I20260811 18:53:23.731283 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:23.729247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:37933 (local address 127.0.0.1:38890)
0811 18:53:23.731015 (+ 1768us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:23.731147 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1629}
W20260811 18:53:23.732400 14021 catalog_manager.cc:4584] TS c932ddc2a3724745ab8f4c97bc178940 (127.27.115.193:37933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75294834aaa74887b60cb9583513c07c: Network error: Client connection negotiation failed: client connection to 127.27.115.193:37933: connect: Connection refused (error 111)
I20260811 18:53:23.738590 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14416
I20260811 18:53:23.749586 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14002
I20260811 18:53:43.760628 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--webserver_interface=127.27.115.254
--webserver_port=42721
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:43.886529 14611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:43.886905 14611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:43.887002 14611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:43.892001 14611 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:53:43.892107 14611 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:53:43.892133 14611 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:53:43.892159 14611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:53:43.892200 14611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:43.892238 14611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:43.892272 14611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:53:43.892306 14611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:53:43.898238 14611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:43801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=42721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:43.899513 14611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:43.900779 14611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:43.911067 14616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:43.911044 14618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:43.911136 14611 server_base.cc:1034] running on GCE node
W20260811 18:53:43.911002 14620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:43.911931 14611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:43.913178 14611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:43.914352 14611 hybrid_clock.cc:648] HybridClock initialized: now 1786474423914321 us; error 50 us; skew 500 ppm
Aug 11 18:53:43 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474423, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:43.917510 14611 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:53:43.918697 14611 webserver.cc:460] Webserver started at http://127.27.115.254:42721/ using document root <none> and password file <none>
I20260811 18:53:43.919313 14611 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:43.919380 14611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:43.923657 14611 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:53:43.926075 14628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:43.927548 14611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:43.927747 14611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "00881dbd851048e3becb22f1defcdaa9"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "34c4b021299822a47edb50bf8a0003f9"
server_key_iv: "aa542464e2dc9686b4cb13fea18bcb0d"
server_key_version: "encryptionkey@0"
I20260811 18:53:43.928409 14611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:43.963974 14611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:43.967805 14611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:43.968039 14611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:43.976348 14611 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:43801
I20260811 18:53:43.976655 14686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:43801 every 8 connection(s)
I20260811 18:53:43.977674 14611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:53:43.980530 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14611
I20260811 18:53:43.981055 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:37933
--local_ip_for_outbound_sockets=127.27.115.193
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=37949
--webserver_interface=127.27.115.193
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:53:43.983232 14687 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:53:43.985090 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap starting.
I20260811 18:53:44.005218 14687 log.cc:824] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:44.017937 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:53:44.018508 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Bootstrap complete.
I20260811 18:53:44.023463 14687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:44.024183 14687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Initialized, Role: FOLLOWER
I20260811 18:53:44.024803 14687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:44.024966 14687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:53:44.025066 14687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:53:44.025205 14687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:53:44.027704 14687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:44.028119 14687 leader_election.cc:302] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00881dbd851048e3becb22f1defcdaa9; no voters:
I20260811 18:53:44.028548 14687 leader_election.cc:288] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:53:44.028689 14692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:53:44.028942 14692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [term 3 LEADER]: Becoming Leader. State: Replica: 00881dbd851048e3becb22f1defcdaa9, State: Running, Role: LEADER
I20260811 18:53:44.029287 14692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } }
I20260811 18:53:44.030057 14687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:53:44.030475 14692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00881dbd851048e3becb22f1defcdaa9. Latest consensus state: current_term: 3 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:44.030611 14692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:44.031113 14693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "00881dbd851048e3becb22f1defcdaa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00881dbd851048e3becb22f1defcdaa9" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 43801 } } }
I20260811 18:53:44.031383 14693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:53:44.031240 14697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:53:44.038236 14697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:53:44.039023 14697 catalog_manager.cc:1259] Loaded cluster ID: 8568f4fa785b4340aec611e5f86acfb8
I20260811 18:53:44.039113 14697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:53:44.041003 14697 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:53:44.041733 14697 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Loaded TSK: 0
I20260811 18:53:44.041837 14697 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00881dbd851048e3becb22f1defcdaa9: Loaded TSK: 1
F20260811 18:53:44.042622 14697 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1012d189ad google::LogMessage::Fail() at ??:0
@ 0x7f1012d1b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1012d184de google::LogMessage::Flush() at ??:0
@ 0x7f1012d19a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f10159dc18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f10159d4f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f101d67d7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f101d65799c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f101d658178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f101d699df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f101d6b882b std::function<>::operator()() at ??:0
@ 0x7f101d657b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f101d658f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f101d655579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f101d699640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1014f6a3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1014f6ac83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1014f6c3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1014f5b512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f101659f6db start_thread at ??:0
@ 0x7f101181661f clone at ??:0
W20260811 18:53:44.135388 14690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:44.135665 14690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:44.135713 14690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:44.140487 14690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:44.140549 14690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:44.140616 14690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:53:44.146970 14690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:37933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=37949
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:44.148190 14690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:44.149530 14690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:44.161902 14737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.162045 14738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.161901 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:44.163049 14690 server_base.cc:1034] running on GCE node
I20260811 18:53:44.163542 14690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:44.164265 14690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:44.165457 14690 hybrid_clock.cc:648] HybridClock initialized: now 1786474424165413 us; error 67 us; skew 500 ppm
Aug 11 18:53:44 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474424, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:44.169435 14690 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:53:44.170751 14690 webserver.cc:460] Webserver started at http://127.27.115.193:37949/ using document root <none> and password file <none>
I20260811 18:53:44.171556 14690 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:44.171656 14690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:44.177340 14690 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:53:44.180639 14752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.182425 14690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:44.182602 14690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "c932ddc2a3724745ab8f4c97bc178940"
format_stamp: "Formatted at 2026-08-11 18:53:01 on dist-test-slave-9jjr"
server_key: "f58dfa19c09ffaf6fbad664f10579a76"
server_key_iv: "801e420f9c4fb049218c08bb0224cd13"
server_key_version: "encryptionkey@0"
I20260811 18:53:44.183409 14690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:44.225956 14690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:44.230746 14690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:44.230981 14690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:44.231804 14690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:44.233387 14760 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:53:44.237986 14690 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:53:44.238098 14690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 18:53:44.238207 14690 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:53:44.240630 14690 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:53:44.240705 14690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260811 18:53:44.241103 14760 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Bootstrap starting.
I20260811 18:53:44.252967 14690 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:37933
I20260811 18:53:44.253427 14880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:37933 every 8 connection(s)
I20260811 18:53:44.254292 14690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:53:44.261389 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14690
I20260811 18:53:44.261955 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:43011
--local_ip_for_outbound_sockets=127.27.115.194
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=46613
--webserver_interface=127.27.115.194
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:53:44.263211 14760 log.cc:824] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Log is configured to *not* fsync() on all Append() calls
I20260811 18:53:44.275009 14760 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:53:44.275586 14760 tablet_bootstrap.cc:492] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Bootstrap complete.
I20260811 18:53:44.277458 14760 ts_tablet_manager.cc:1397] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Time spent bootstrapping tablet: real 0.037s user 0.029s sys 0.003s
I20260811 18:53:44.281172 14760 raft_consensus.cc:348] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:44.281805 14760 raft_consensus.cc:729] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c932ddc2a3724745ab8f4c97bc178940, State: Initialized, Role: FOLLOWER
I20260811 18:53:44.282410 14760 consensus_queue.cc:260] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:44.283635 14760 ts_tablet_manager.cc:1428] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
W20260811 18:53:44.334404 14884 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:53:44.256930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:51379)
0811 18:53:44.257723 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.257805 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:44.259120 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:44.333294 (+ 74174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":191,"threads_started":1}
W20260811 18:53:44.335124 14881 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:43801 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:43801: Client connection negotiation failed: client connection to 127.27.115.254:43801: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:53:44.336179 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.335826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:59191)
0811 18:53:44.336000 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.336096 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:53:44.337028 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.336748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:44991)
0811 18:53:44.336890 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.336948 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 18:53:44.337270 14881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:53:44.401307 14887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:44.401718 14887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:44.401811 14887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:44.406178 14887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:44.406261 14887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:44.406323 14887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:53:44.412375 14887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:43011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=46613
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:44.413637 14887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:44.414808 14887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:44.426637 14893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.426637 14896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.426637 14894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:44.427076 14887 server_base.cc:1034] running on GCE node
I20260811 18:53:44.427743 14887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:44.428603 14887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:44.432041 14887 hybrid_clock.cc:648] HybridClock initialized: now 1786474424431850 us; error 193 us; skew 500 ppm
Aug 11 18:53:44 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474424, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:44.435791 14887 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:53:44.437018 14887 webserver.cc:460] Webserver started at http://127.27.115.194:46613/ using document root <none> and password file <none>
I20260811 18:53:44.437747 14887 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:44.437861 14887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:44.442822 14887 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:44.445590 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.447257 14887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:44.447445 14887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "b30a430e35154648aadf335ae1913c16"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "b8cbc5d1bfec76a9e1ac8fe04b234d3e"
server_key_iv: "12c0a73679317e988983d385482fb8aa"
server_key_version: "encryptionkey@0"
I20260811 18:53:44.448107 14887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:44.460144 14887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:44.463568 14887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:44.463788 14887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:44.464438 14887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:44.465354 14887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:44.465431 14887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.465507 14887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:44.465557 14887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.475287 14887 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:43011
I20260811 18:53:44.475312 15032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:43011 every 8 connection(s)
I20260811 18:53:44.476445 14887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:53:44.480432 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.478735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:56019)
0811 18:53:44.479286 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.479722 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":121,"threads_started":1}
W20260811 18:53:44.480885 15033 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:43801 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:43801: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
I20260811 18:53:44.481514 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.481288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:37871)
0811 18:53:44.481406 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.481462 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:53:44.482164 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.481915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:60877)
0811 18:53:44.482022 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.482067 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:53:44.482366 15033 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:53:44.482767 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 14887
I20260811 18:53:44.483217 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:33179
--local_ip_for_outbound_sockets=127.27.115.195
--tserver_master_addrs=127.27.115.254:43801
--webserver_port=42841
--webserver_interface=127.27.115.195
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:53:44.603746 15038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:53:44.604084 15038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:53:44.604131 15038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:53:44.609475 15038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:53:44.609606 15038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:53:44.609714 15038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:53:44.615808 15038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:33179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=42841
--tserver_master_addrs=127.27.115.254:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:53:44.616984 15038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:53:44.618023 15038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:53:44.629678 15046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.629738 15043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:53:44.629678 15044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:53:44.630031 15038 server_base.cc:1034] running on GCE node
I20260811 18:53:44.630582 15038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:53:44.631362 15038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:53:44.632575 15038 hybrid_clock.cc:648] HybridClock initialized: now 1786474424632557 us; error 30 us; skew 500 ppm
Aug 11 18:53:44 dist-test-slave-9jjr krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474424, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:53:44.635921 15038 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:53:44.637048 15038 webserver.cc:460] Webserver started at http://127.27.115.195:42841/ using document root <none> and password file <none>
I20260811 18:53:44.637714 15038 fs_manager.cc:359] Metadata directory not provided
I20260811 18:53:44.637800 15038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:53:44.642654 15038 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:53:44.645423 15053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.647161 15038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:53:44.647359 15038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2ea3bc53fa1c45dc82bc2d961ad86073"
format_stamp: "Formatted at 2026-08-11 18:53:02 on dist-test-slave-9jjr"
server_key: "66fe4db7e0c4e99676315bfdf04fd105"
server_key_iv: "94323409c3163e67adb03dcf89976b7a"
server_key_version: "encryptionkey@0"
I20260811 18:53:44.648061 15038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:53:44.670764 15038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:53:44.674582 15038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:53:44.674822 15038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:53:44.675500 15038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:53:44.676451 15038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:53:44.676528 15038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.676602 15038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:53:44.676640 15038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:53:44.687096 15038 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:33179
I20260811 18:53:44.687129 15174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:33179 every 8 connection(s)
I20260811 18:53:44.688210 15038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:53:44.691829 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15038
I20260811 18:53:44.692785 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.690819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:37359)
0811 18:53:44.691394 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.691896 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":110,"threads_started":1}
W20260811 18:53:44.693297 15175 heartbeater.cc:644] Failed to heartbeat to 127.27.115.254:43801 (0 consecutive failures): Network error: Failed to ping master at 127.27.115.254:43801: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
I20260811 18:53:44.693945 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.693678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:51281)
0811 18:53:44.693820 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.693874 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 18:53:44.694697 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:44.694439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:41879)
0811 18:53:44.694597 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:44.694645 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 18:53:44.694945 15175 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:53:45.338864 15180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.338243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:49395)
0811 18:53:45.338571 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:45.338642 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":86,"threads_started":1}
I20260811 18:53:45.483492 15181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.482982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:48257)
0811 18:53:45.483269 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:45.483335 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
I20260811 18:53:45.696048 15182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.695595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:44739)
0811 18:53:45.695873 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:45.695937 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260811 18:53:45.810817 15184 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:45.811141 15184 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:45.812305 15184 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
Aug 11 18:53:45 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474424, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.194@KRBTEST.COM
Aug 11 18:53:45 dist-test-slave-9jjr krb5kdc[13384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474424, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.195@KRBTEST.COM
I20260811 18:53:45.834848 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.812782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:46221 (local address 127.27.115.194:43011)
0811 18:53:45.813134 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:45.813157 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:45.813294 (+ 137us) server_negotiation.cc:407] Connection header received
0811 18:53:45.814259 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:45.814266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:45.814551 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:45.814849 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:45.816049 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.817052 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818301 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.818792 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818982 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:45.829006 (+ 10024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:45.829012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:45.829026 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:45.829065 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:45.831174 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832077 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:45.832084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:45.832090 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:45.832256 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832764 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:45.832767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:45.832769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:45.833389 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:45.833539 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:45.834405 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 18:53:45.834671 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
I20260811 18:53:45.834915 15185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.812618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:43011 (local address 127.27.115.193:46221)
0811 18:53:45.813154 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:45.813168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:45.813282 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:45.814798 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:45.814801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:45.815348 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:45.815852 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:45.815867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.817229 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.817233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:45.818109 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:45.818172 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.818933 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:45.819135 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:45.826631 (+ 7496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:45.826670 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:45.828651 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:45.831351 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:45.831358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:45.831373 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:45.831920 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:45.832374 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:45.832380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:45.832626 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:45.833549 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:45.833560 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:45.833967 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 18:53:45.834244 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 18:53:45.834664 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":128,"spinlock_wait_cycles":40448,"thread_start_us":99,"threads_started":1}
I20260811 18:53:45.834915 15180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.812548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:33179 (local address 127.27.115.193:47751)
0811 18:53:45.812746 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:45.812764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:53:45.812877 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:53:45.814746 (+ 1869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:53:45.814752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:53:45.815347 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:53:45.815851 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:45.815867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.817460 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.817465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:45.818110 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:53:45.818121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.818937 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:53:45.819144 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:45.826632 (+ 7488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:53:45.826661 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:53:45.828909 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:53:45.831478 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:45.831481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:45.831482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:45.831920 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:45.832374 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:53:45.832380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:53:45.832626 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:53:45.833549 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:53:45.833557 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:53:45.833967 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 18:53:45.834244 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 18:53:45.834720 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":330,"spinlock_wait_cycles":38528}
I20260811 18:53:45.835809 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:45.812763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:47751 (local address 127.27.115.195:33179)
0811 18:53:45.813028 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 18:53:45.813056 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 18:53:45.813104 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:53:45.813896 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:53:45.813903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:53:45.814363 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:53:45.814698 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:53:45.816054 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.817297 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818298 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:53:45.818789 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:53:45.818986 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:53:45.829190 (+ 10204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:53:45.829196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:53:45.829209 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:53:45.829246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:53:45.831341 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832079 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:45.832093 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:45.832098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:45.832256 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:53:45.832764 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:53:45.832767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:53:45.832769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:53:45.833388 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:53:45.833546 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:53:45.835293 (+ 1747us) server_negotiation.cc:299] Negotiation successful
0811 18:53:45.835597 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
W20260811 18:53:45.838260 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:45.838449 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:45.838533 14756 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:45.838776 15184 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:46.340428 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:46.339918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:34101)
0811 18:53:46.340227 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:46.340318 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
I20260811 18:53:46.484884 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:46.484426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:38285)
0811 18:53:46.484719 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:46.484783 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260811 18:53:46.697556 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:46.696975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:40029)
0811 18:53:46.697349 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:46.697450 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":95,"threads_started":1}
I20260811 18:53:47.342073 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:47.341546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:40981)
0811 18:53:47.341890 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:47.341970 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
I20260811 18:53:47.375022 15192 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:47.375176 15192 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:47.375751 15192 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
W20260811 18:53:47.376894 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:47.377018 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:47.377092 14756 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:47.377285 15192 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:47.486333 15193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:47.485792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:39149)
0811 18:53:47.486158 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:47.486222 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":94,"threads_started":1}
I20260811 18:53:47.699543 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:47.698867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:41739)
0811 18:53:47.699286 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:47.699376 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260811 18:53:48.343622 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:48.343152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:46613)
0811 18:53:48.343453 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:48.343520 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260811 18:53:48.487727 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:48.487260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:35833)
0811 18:53:48.487562 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:48.487629 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":90,"threads_started":1}
I20260811 18:53:48.700940 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:48.700429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:45777)
0811 18:53:48.700771 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:48.700839 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":90,"threads_started":1}
I20260811 18:53:49.188871 15198 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:49.189030 15198 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:49.189640 15198 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
W20260811 18:53:49.190649 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:49.190800 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:49.190914 14756 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:49.191123 15198 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:49.345440 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:49.344880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:54805)
0811 18:53:49.345242 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:49.345325 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
I20260811 18:53:49.489372 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:49.488684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:58847)
0811 18:53:49.489186 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:49.489255 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":87,"threads_started":1}
I20260811 18:53:49.702567 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:49.701992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:48685)
0811 18:53:49.702386 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:49.702453 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":194,"threads_started":1}
I20260811 18:53:50.346997 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:50.346483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:54297)
0811 18:53:50.346823 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:50.346893 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":91,"threads_started":1}
I20260811 18:53:50.490880 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:50.490390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:60503)
0811 18:53:50.490696 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:50.490765 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260811 18:53:50.704103 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:50.703597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:44227)
0811 18:53:50.703937 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:50.704004 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260811 18:53:51.348449 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:51.347910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:59701)
0811 18:53:51.348206 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:51.348288 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
I20260811 18:53:51.492385 15206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:51.491853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:60495)
0811 18:53:51.492211 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:51.492278 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":81,"threads_started":1}
I20260811 18:53:51.705651 15207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:51.705132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:48209)
0811 18:53:51.705496 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:51.705552 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260811 18:53:52.147938 15209 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:52.148095 15209 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:52.148687 15209 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
W20260811 18:53:52.149606 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:52.149750 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:52.149874 14754 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:52.150053 15209 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:52.349941 15210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:52.349407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:39685)
0811 18:53:52.349763 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:52.349835 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":101,"threads_started":1}
I20260811 18:53:52.493903 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:52.493401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:55687)
0811 18:53:52.493708 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:52.493785 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
I20260811 18:53:52.707172 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:52.706682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:57011)
0811 18:53:52.706978 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:52.707048 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260811 18:53:53.351693 15213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:53.351118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:57987)
0811 18:53:53.351426 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:53.351493 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
I20260811 18:53:53.495196 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:53.494716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:41365)
0811 18:53:53.495034 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:53.495091 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":103,"threads_started":1}
I20260811 18:53:53.708655 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:53.708195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:47287)
0811 18:53:53.708489 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:53.708550 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260811 18:53:54.353434 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:54.352810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:36259)
0811 18:53:54.353154 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:54.353223 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":81,"threads_started":1}
I20260811 18:53:54.496706 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:54.496208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:49019)
0811 18:53:54.496534 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:54.496606 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260811 18:53:54.710474 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:54.709818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:42693)
0811 18:53:54.710241 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:54.710347 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":94,"threads_started":1}
I20260811 18:53:55.355008 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:55.354501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:55405)
0811 18:53:55.354830 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:55.354901 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":94,"threads_started":1}
I20260811 18:53:55.498071 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:55.497620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:51535)
0811 18:53:55.497889 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:55.497947 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260811 18:53:55.712038 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:55.711423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:38917)
0811 18:53:55.711868 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:55.711936 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":137,"threads_started":1}
I20260811 18:53:56.356297 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:56.355813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:53979)
0811 18:53:56.356127 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:56.356193 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":86,"threads_started":1}
I20260811 18:53:56.499699 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:56.499031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:45191)
0811 18:53:56.499486 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:56.499567 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":135,"threads_started":1}
I20260811 18:53:56.713507 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:56.712988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:46437)
0811 18:53:56.713316 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:56.713389 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":112,"threads_started":1}
I20260811 18:53:57.357921 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:57.357374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:33175)
0811 18:53:57.357750 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:57.357819 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":95,"threads_started":1}
I20260811 18:53:57.501150 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:57.500620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:51469)
0811 18:53:57.500942 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:57.501012 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":104,"threads_started":1}
I20260811 18:53:57.628170 15228 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:57.628324 15228 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:57.629063 15228 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
W20260811 18:53:57.629981 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:57.630103 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:57.630215 14756 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:57.630394 15228 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:57.715073 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:57.714594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:44107)
0811 18:53:57.714901 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:57.714970 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":87,"threads_started":1}
I20260811 18:53:58.359548 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:58.358999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:46349)
0811 18:53:58.359363 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:58.359440 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":140,"threads_started":1}
I20260811 18:53:58.502590 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:58.502080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:50597)
0811 18:53:58.502431 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:58.502492 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":161,"threads_started":1}
I20260811 18:53:58.716715 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:58.716190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.195:49709)
0811 18:53:58.716525 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:58.716595 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":96,"threads_started":1}
I20260811 18:53:59.361074 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:59.360620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.193:48297)
0811 18:53:59.360918 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:59.360978 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260811 18:53:59.459870 15234 raft_consensus.cc:482] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:53:59.460024 15234 raft_consensus.cc:504] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea3bc53fa1c45dc82bc2d961ad86073" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 33179 } } peers { permanent_uuid: "b30a430e35154648aadf335ae1913c16" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 43011 } } peers { permanent_uuid: "c932ddc2a3724745ab8f4c97bc178940" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 37933 } }
I20260811 18:53:59.460433 15234 leader_election.cc:288] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179), b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011)
W20260811 18:53:59.461385 14756 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea3bc53fa1c45dc82bc2d961ad86073 (127.27.115.195:33179): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
W20260811 18:53:59.461522 14754 leader_election.cc:341] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b30a430e35154648aadf335ae1913c16 (127.27.115.194:43011): Not found: Tablet not found: 75294834aaa74887b60cb9583513c07c
I20260811 18:53:59.461632 14754 leader_election.cc:302] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c932ddc2a3724745ab8f4c97bc178940; no voters: 2ea3bc53fa1c45dc82bc2d961ad86073, b30a430e35154648aadf335ae1913c16
I20260811 18:53:59.461840 15234 raft_consensus.cc:2726] T 75294834aaa74887b60cb9583513c07c P c932ddc2a3724745ab8f4c97bc178940 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:53:59.504237 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:53:59.503720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:43801 (local address 127.27.115.194:42777)
0811 18:53:59.504060 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:53:59.504135 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.254:43801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":129,"threads_started":1}
I20260811 18:53:59.693508 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14690
I20260811 18:53:59.703653 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 14887
I20260811 18:53:59.714109 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15038
2026-08-11T18:53:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59750 ms)
[----------] 30 tests from SecurityITest (265996 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:53:59 dist-test-slave-9jjr krb5kdc[15238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:53:59 dist-test-slave-9jjr krb5kdc[15238](info): set up 2 sockets
Aug 11 18:53:59 dist-test-slave-9jjr krb5kdc[15238](info): commencing operation
krb5kdc: starting...
W20260811 18:54:00.853466 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:00 dist-test-slave-9jjr krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:00.995512 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35467
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:35467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:01.111886 15253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:01.112267 15253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:01.112365 15253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:01.116465 15253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:01.116567 15253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:01.116601 15253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:01.116640 15253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:01.116676 15253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:01.122435 15253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:35467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:35467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:01.123590 15253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:01.124642 15253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:01.133894 15259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:01.133939 15253 server_base.cc:1034] running on GCE node
W20260811 18:54:01.133869 15260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.133898 15262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:01.134923 15253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:01.136029 15253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:01.137240 15253 hybrid_clock.cc:648] HybridClock initialized: now 1786474441137218 us; error 35 us; skew 500 ppm
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:01.140455 15253 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:01.141481 15253 webserver.cc:460] Webserver started at http://127.27.115.254:43299/ using document root <none> and password file <none>
I20260811 18:54:01.142045 15253 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:01.142118 15253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:01.142374 15253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:01.144662 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "0ffff12c438c486baea90821a083c647"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "add6ee6958be6347abd3bf84a6ff277a"
server_key_iv: "08bf0a738b473510a92ffca799264619"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.145287 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "0ffff12c438c486baea90821a083c647"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "add6ee6958be6347abd3bf84a6ff277a"
server_key_iv: "08bf0a738b473510a92ffca799264619"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.149379 15253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 18:54:01.152244 15270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.153806 15253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:01.153982 15253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "0ffff12c438c486baea90821a083c647"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "add6ee6958be6347abd3bf84a6ff277a"
server_key_iv: "08bf0a738b473510a92ffca799264619"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.154124 15253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:01.176719 15253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:01.180478 15253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:01.180703 15253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:01.188709 15253 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:35467
I20260811 18:54:01.188735 15327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:35467 every 8 connection(s)
I20260811 18:54:01.189752 15253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:01.192587 15328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:01.193665 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15253
I20260811 18:54:01.193822 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:01.194159 28111 external_mini_cluster.cc:1428] Setting key 87fcc4437294496d81f995ae8cd50d50
I20260811 18:54:01.199801 15328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: Bootstrap starting.
I20260811 18:54:01.202881 15328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:01.203852 15328 log.cc:824] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:01.206935 15328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: No bootstrap required, opened a new log
I20260811 18:54:01.212147 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.195711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.27.115.254:35467)
0811 18:54:01.196192 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:01.196219 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:01.196263 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:54:01.197100 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:01.197124 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:01.197574 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:01.197957 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:01.198908 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.200353 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.201320 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.202043 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.202277 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:01.205593 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:01.205601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:01.205621 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:01.205657 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:01.208248 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.208810 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.208817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.208824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.208989 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.209354 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.209358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.209359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.210034 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:01.210284 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:01.210681 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:54:01.211031 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":123,"threads_started":1}
I20260811 18:54:01.212160 15328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } }
I20260811 18:54:01.212491 15328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:01.212551 15328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffff12c438c486baea90821a083c647, State: Initialized, Role: FOLLOWER
I20260811 18:54:01.213060 15328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } }
I20260811 18:54:01.213231 15328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:01.213292 15328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:01.213425 15328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:01.214622 15328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } }
I20260811 18:54:01.214987 15328 leader_election.cc:302] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ffff12c438c486baea90821a083c647; no voters:
I20260811 18:54:01.215317 15328 leader_election.cc:288] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:01.215503 15333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:01.215772 15333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [term 1 LEADER]: Becoming Leader. State: Replica: 0ffff12c438c486baea90821a083c647, State: Running, Role: LEADER
I20260811 18:54:01.216187 15333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } }
I20260811 18:54:01.216706 15328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:01.218076 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ffff12c438c486baea90821a083c647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } } }
I20260811 18:54:01.218365 15335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ffff12c438c486baea90821a083c647. Latest consensus state: current_term: 1 leader_uuid: "0ffff12c438c486baea90821a083c647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffff12c438c486baea90821a083c647" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 35467 } } }
I20260811 18:54:01.218385 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:01.218575 15335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:01.219111 15343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:01.222700 15343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:01.229653 15343 catalog_manager.cc:1347] Generated new cluster ID: 9c0f00e55987472092effce1f26aaed7
I20260811 18:54:01.229755 15343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:01.244109 15343 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:01.245236 15343 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:01.252806 15343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: Generated new TSK 0
I20260811 18:54:01.253810 15343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:54:01.290287 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:01.417948 15354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:01.418311 15354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:01.418418 15354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:01.422525 15354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:01.422598 15354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:01.422659 15354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:01.428444 15354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:01.429571 15354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:01.430631 15354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:01.441530 15360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.441531 15362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.441728 15359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:01.441996 15354 server_base.cc:1034] running on GCE node
I20260811 18:54:01.442575 15354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:01.443364 15354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:01.444585 15354 hybrid_clock.cc:648] HybridClock initialized: now 1786474441444566 us; error 31 us; skew 500 ppm
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:01.451558 15354 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:54:01.452626 15354 webserver.cc:460] Webserver started at http://127.27.115.193:36531/ using document root <none> and password file <none>
I20260811 18:54:01.453248 15354 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:01.453317 15354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:01.453503 15354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:01.455917 15354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "b7f119a0968247d9805e8a9cf256c5f8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "81ac77a0657bbd9102d02e0b48eef337"
server_key_iv: "4bb957cc931628b11ac14d689e8088f2"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.456557 15354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "b7f119a0968247d9805e8a9cf256c5f8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "81ac77a0657bbd9102d02e0b48eef337"
server_key_iv: "4bb957cc931628b11ac14d689e8088f2"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.461002 15354 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 18:54:01.463843 15372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.465301 15354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:54:01.465467 15354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "b7f119a0968247d9805e8a9cf256c5f8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "81ac77a0657bbd9102d02e0b48eef337"
server_key_iv: "4bb957cc931628b11ac14d689e8088f2"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.465602 15354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:01.479769 15354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:01.483558 15354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:01.483791 15354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:01.484472 15354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:01.485373 15354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:01.485422 15354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.485462 15354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:01.485519 15354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.496680 15354 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:44093
I20260811 18:54:01.496729 15496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:44093 every 8 connection(s)
I20260811 18:54:01.498162 15354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:01.499118 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15354
I20260811 18:54:01.499293 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:01.499599 28111 external_mini_cluster.cc:1428] Setting key ab865d8a4f5197bb28fa042162c4d91d
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:01.532811 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.501598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:53825 (local address 127.27.115.254:35467)
0811 18:54:01.502305 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:01.502312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:01.502859 (+ 547us) server_negotiation.cc:407] Connection header received
0811 18:54:01.504786 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:01.504795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:01.504872 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:01.504989 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:01.507203 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.508018 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.510011 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.510446 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.510540 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:01.517331 (+ 6791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:01.517339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:01.517344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:01.517407 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:01.529290 (+ 11883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.530369 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.530375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.530380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.530505 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.531066 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.531128 (+ 62us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.531133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.531374 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:01.531464 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:01.532400 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 18:54:01.532561 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577}
I20260811 18:54:01.533838 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.502148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35467 (local address 127.27.115.193:53825)
0811 18:54:01.502695 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:01.502753 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:01.504488 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:01.505475 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:01.505487 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:01.506214 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:01.506980 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:01.507003 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.508230 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.508233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:01.509827 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:01.509844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.510845 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.510852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:01.511154 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:01.512029 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:01.512063 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:01.516977 (+ 4914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:01.529545 (+ 12568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:01.529559 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:01.529584 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:01.530202 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:01.530650 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:01.530653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:01.530655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:01.530913 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:01.531504 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:01.531514 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:01.531961 (+ 447us) client_negotiation.cc:769] Sending connection context
0811 18:54:01.532231 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 18:54:01.532762 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":153,"threads_started":1}
I20260811 18:54:01.535504 15497 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35467
I20260811 18:54:01.535811 15497 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:01.536558 15497 heartbeater.cc:511] Master 127.27.115.254:35467 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:54:01.539940 15289 ts_manager.cc:194] Registered new tserver with Master: b7f119a0968247d9805e8a9cf256c5f8 (127.27.115.193:44093)
I20260811 18:54:01.540830 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:54:01.541718 15289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:53825
W20260811 18:54:01.663637 15502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:01.664044 15502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:01.664127 15502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:01.668442 15502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:01.668550 15502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:01.668656 15502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:01.674451 15502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:01.675693 15502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:01.676795 15502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:01.688210 15508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.688215 15510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.688290 15507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:01.688445 15502 server_base.cc:1034] running on GCE node
I20260811 18:54:01.689035 15502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:01.689752 15502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:01.690945 15502 hybrid_clock.cc:648] HybridClock initialized: now 1786474441690926 us; error 32 us; skew 500 ppm
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:01.693771 15502 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:54:01.694778 15502 webserver.cc:460] Webserver started at http://127.27.115.194:37779/ using document root <none> and password file <none>
I20260811 18:54:01.695369 15502 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:01.695464 15502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:01.695719 15502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:01.697840 15502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "c073820394884aa1a03dfcdc6bebacd8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "b8d95826897d697330b3ed95d8557be1"
server_key_iv: "92fd6e75029d90a9f77d9a9badb55beb"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.698446 15502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "c073820394884aa1a03dfcdc6bebacd8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "b8d95826897d697330b3ed95d8557be1"
server_key_iv: "92fd6e75029d90a9f77d9a9badb55beb"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.702795 15502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:01.705837 15521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.707551 15502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:01.707760 15502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "c073820394884aa1a03dfcdc6bebacd8"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "b8d95826897d697330b3ed95d8557be1"
server_key_iv: "92fd6e75029d90a9f77d9a9badb55beb"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.707912 15502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:01.722673 15502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:01.726727 15502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:01.726977 15502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:01.727749 15502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:01.728778 15502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:01.728871 15502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.728976 15502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:01.729028 15502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.739670 15502 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:32809
I20260811 18:54:01.739809 15641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:32809 every 8 connection(s)
I20260811 18:54:01.740715 15502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:01.750494 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15502
I20260811 18:54:01.750660 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:01.750967 28111 external_mini_cluster.cc:1428] Setting key 92f3720ca35743591a99c7bff27f51cb
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:01.757407 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.742851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:54719 (local address 127.27.115.254:35467)
0811 18:54:01.743026 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:01.743031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:01.744180 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 18:54:01.745868 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:01.745872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:01.745956 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:01.746044 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:01.747428 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.747974 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.749198 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.749432 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.749553 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:01.752884 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:01.752888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:01.752890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:01.752936 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:01.754768 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.755454 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.755460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.755462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.755569 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:01.755975 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:01.755981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:01.755985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:01.756267 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:01.756374 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:01.757108 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 18:54:01.757231 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:54:01.758801 15645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.743219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35467 (local address 127.27.115.194:54719)
0811 18:54:01.743965 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:01.744032 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:01.745590 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:01.746332 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:01.746340 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:01.746738 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:01.747260 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:01.747275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.748125 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.748129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:01.749046 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:01.749056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:01.749610 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:01.749614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:01.749800 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:01.750545 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:01.750567 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:01.752683 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:01.754989 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:01.755000 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:01.755024 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:01.755338 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:01.755688 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:01.755690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:01.755691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:01.755865 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:01.756408 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:01.756420 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:01.756816 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 18:54:01.757056 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 18:54:01.757467 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":135,"threads_started":1}
I20260811 18:54:01.760215 15642 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35467
I20260811 18:54:01.760504 15642 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:01.761122 15642 heartbeater.cc:511] Master 127.27.115.254:35467 requested a full tablet report, sending...
I20260811 18:54:01.762460 15289 ts_manager.cc:194] Registered new tserver with Master: c073820394884aa1a03dfcdc6bebacd8 (127.27.115.194:32809)
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:54:01.763155 15289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:54719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:54:01.786868 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:01.911454 15648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:01.911865 15648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:01.911962 15648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:01.917011 15648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:01.917114 15648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:01.917209 15648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:01.923364 15648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:35467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:01.924706 15648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:01.925825 15648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:01.936578 15653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.936609 15656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:01.936609 15654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:01.937059 15648 server_base.cc:1034] running on GCE node
I20260811 18:54:01.937696 15648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:01.938575 15648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:01.939791 15648 hybrid_clock.cc:648] HybridClock initialized: now 1786474441939768 us; error 40 us; skew 500 ppm
Aug 11 18:54:01 dist-test-slave-9jjr krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:01.943346 15648 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:54:01.944540 15648 webserver.cc:460] Webserver started at http://127.27.115.195:46389/ using document root <none> and password file <none>
I20260811 18:54:01.945217 15648 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:01.945289 15648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:01.945539 15648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:01.948118 15648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "2e6531e461f34217a42d4b5c14e9129c"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "9dcda92cf6c6ae71eff475be0ea44a21"
server_key_iv: "7e00fad36a0c9812e929b22b2b13c897"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.948849 15648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "2e6531e461f34217a42d4b5c14e9129c"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "9dcda92cf6c6ae71eff475be0ea44a21"
server_key_iv: "7e00fad36a0c9812e929b22b2b13c897"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.953336 15648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:01.956485 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.958097 15648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:54:01.958328 15648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "2e6531e461f34217a42d4b5c14e9129c"
format_stamp: "Formatted at 2026-08-11 18:54:01 on dist-test-slave-9jjr"
server_key: "9dcda92cf6c6ae71eff475be0ea44a21"
server_key_iv: "7e00fad36a0c9812e929b22b2b13c897"
server_key_version: "encryptionkey@0"
I20260811 18:54:01.958469 15648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:01.977948 15648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:01.982317 15648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:01.982592 15648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:01.983361 15648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:01.984388 15648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:01.984441 15648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.984515 15648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:01.984565 15648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:01.995150 15648 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:35861
I20260811 18:54:01.995172 15790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:35861 every 8 connection(s)
I20260811 18:54:01.996194 15648 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:01.996613 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15648
I20260811 18:54:01.996780 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:01.997184 28111 external_mini_cluster.cc:1428] Setting key b7e78306dcec845bc5de5f94248e600b
Aug 11 18:54:02 dist-test-slave-9jjr krb5kdc[15238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474441, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:02.013834 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.998391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:56483 (local address 127.27.115.254:35467)
0811 18:54:01.998539 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:01.998544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:01.999914 (+ 1370us) server_negotiation.cc:407] Connection header received
0811 18:54:02.001487 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.001490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.001545 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.001648 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:02.003373 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.003936 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.005367 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.005623 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.005741 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.009055 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:02.009059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:02.009062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:02.009096 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:02.011021 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:02.011853 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:02.011858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:02.011859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:02.012019 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:02.012447 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:02.012451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:02.012452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:02.012703 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:02.012822 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.013494 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.013664 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:54:02.014945 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:01.998984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:35467 (local address 127.27.115.195:56483)
0811 18:54:01.999714 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:01.999782 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.001222 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.001849 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.001858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.002540 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:02.003181 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.003206 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.004178 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.004182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.005189 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.005202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.005847 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.005851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.006047 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.006940 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:02.006960 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:02.008795 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:02.011291 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:02.011298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:02.011317 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:02.011728 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:02.012131 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:02.012134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:02.012138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:02.012322 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:02.012824 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:02.012834 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:02.013219 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.013446 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.013850 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":226,"threads_started":1}
I20260811 18:54:02.016511 15791 heartbeater.cc:347] Connected to a master server at 127.27.115.254:35467
I20260811 18:54:02.016887 15791 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:02.017582 15791 heartbeater.cc:511] Master 127.27.115.254:35467 requested a full tablet report, sending...
I20260811 18:54:02.019129 15289 ts_manager.cc:194] Registered new tserver with Master: 2e6531e461f34217a42d4b5c14e9129c (127.27.115.195:35861)
I20260811 18:54:02.019840 15289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:56483
I20260811 18:54:02.025084 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:02.038642 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.027880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.27.115.254:35467)
0811 18:54:02.028059 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.028063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.028311 (+ 248us) server_negotiation.cc:407] Connection header received
0811 18:54:02.028566 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.028570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.028625 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.028704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:02.029735 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.030672 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.031841 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.032103 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.032212 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.034083 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:02.034087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:02.034090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:02.034191 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:02.035951 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:02.036565 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:02.036568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:02.036569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:02.036699 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:02.037115 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:02.037119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:02.037120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:02.037433 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:02.037559 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.038302 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.038449 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:54:02.042913 15289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:02.046471 15289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:02.065269 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.058211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:32809 (local address 127.0.0.1:50774)
0811 18:54:02.058943 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.058957 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.059053 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.059829 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.059832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.059845 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:02.060105 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.060114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061894 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.061898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.063124 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.063137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.064564 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.064568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.064708 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.064759 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.064899 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.065065 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":82,"thread_start_us":114,"threads_started":1}
I20260811 18:54:02.065448 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.057932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:35861 (local address 127.0.0.1:42574)
0811 18:54:02.058398 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.058463 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.058618 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.059739 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.059748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.059783 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:02.060105 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.060114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061804 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.061807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.063669 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.063680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.064878 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.064882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.064977 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.065001 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.065107 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.065211 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":645,"thread_start_us":103,"threads_started":1}
I20260811 18:54:02.065629 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.058932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.27.115.194:32809)
0811 18:54:02.059343 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.059354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.059382 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:54:02.059463 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.059468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.059660 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.060258 (+ 598us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:02.060425 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061670 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.063348 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.064402 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.064562 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.064734 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.065155 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.065278 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":145,"threads_started":1}
I20260811 18:54:02.066063 15809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.058860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:44093 (local address 127.0.0.1:54002)
0811 18:54:02.059402 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.059418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.059519 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.060375 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.060381 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.060472 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:02.060547 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.060553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061591 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.061595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.064051 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.064060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.065492 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.065496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.065589 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.065666 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.065805 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.065918 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":679,"thread_start_us":107,"threads_started":1}
I20260811 18:54:02.066213 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.058164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.27.115.195:35861)
0811 18:54:02.058708 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.058722 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.058749 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:54:02.058840 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.058847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.059507 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.060257 (+ 750us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:02.060425 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061640 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.063809 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.064664 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.064895 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.065036 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.065889 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.066023 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":186,"threads_started":1}
I20260811 18:54:02.066593 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.059114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.27.115.193:44093)
0811 18:54:02.059684 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.059691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.059717 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:54:02.059860 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.059865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.060043 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.060209 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:02.060683 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.061442 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.064195 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.065064 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.065263 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.065418 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.066269 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.066421 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
I20260811 18:54:02.069568 15573 tablet_service.cc:1467] Processing CreateTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 (DEFAULT_TABLE table=test-table [id=09e7818e7cf346579a226a99b13c90ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:02.069713 15721 tablet_service.cc:1467] Processing CreateTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 (DEFAULT_TABLE table=test-table [id=09e7818e7cf346579a226a99b13c90ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:02.069818 15425 tablet_service.cc:1467] Processing CreateTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 (DEFAULT_TABLE table=test-table [id=09e7818e7cf346579a226a99b13c90ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:02.071089 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0bdbbde36c4ec89696fed74e42cf72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:02.071100 15721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0bdbbde36c4ec89696fed74e42cf72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:02.071100 15425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0bdbbde36c4ec89696fed74e42cf72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:02.080984 15813 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Bootstrap starting.
I20260811 18:54:02.081876 15812 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Bootstrap starting.
I20260811 18:54:02.081867 15814 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Bootstrap starting.
I20260811 18:54:02.084270 15814 tablet_bootstrap.cc:654] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:02.084264 15813 tablet_bootstrap.cc:654] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:02.084375 15812 tablet_bootstrap.cc:654] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:02.085373 15814 log.cc:824] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:02.085860 15813 log.cc:824] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:02.086180 15812 log.cc:824] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:02.088053 15814 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: No bootstrap required, opened a new log
I20260811 18:54:02.088362 15814 ts_tablet_manager.cc:1397] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:54:02.088549 15813 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: No bootstrap required, opened a new log
I20260811 18:54:02.088546 15812 tablet_bootstrap.cc:492] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: No bootstrap required, opened a new log
I20260811 18:54:02.088762 15813 ts_tablet_manager.cc:1397] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:54:02.088762 15812 ts_tablet_manager.cc:1397] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:54:02.092484 15814 raft_consensus.cc:348] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.092885 15814 raft_consensus.cc:374] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:02.092948 15814 raft_consensus.cc:729] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f119a0968247d9805e8a9cf256c5f8, State: Initialized, Role: FOLLOWER
I20260811 18:54:02.093001 15813 raft_consensus.cc:348] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.093302 15813 raft_consensus.cc:374] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:02.093390 15813 raft_consensus.cc:729] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6531e461f34217a42d4b5c14e9129c, State: Initialized, Role: FOLLOWER
I20260811 18:54:02.093003 15812 raft_consensus.cc:348] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.093535 15814 consensus_queue.cc:260] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.093643 15812 raft_consensus.cc:374] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:02.093698 15812 raft_consensus.cc:729] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c073820394884aa1a03dfcdc6bebacd8, State: Initialized, Role: FOLLOWER
I20260811 18:54:02.093820 15813 consensus_queue.cc:260] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.094165 15812 consensus_queue.cc:260] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.094826 15814 ts_tablet_manager.cc:1428] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:54:02.094877 15813 ts_tablet_manager.cc:1428] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 18:54:02.095063 15812 ts_tablet_manager.cc:1428] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 18:54:02.095274 15791 heartbeater.cc:503] Master 127.27.115.254:35467 was elected leader, sending a full tablet report...
I20260811 18:54:02.095276 15497 heartbeater.cc:503] Master 127.27.115.254:35467 was elected leader, sending a full tablet report...
I20260811 18:54:02.095623 15642 heartbeater.cc:503] Master 127.27.115.254:35467 was elected leader, sending a full tablet report...
W20260811 18:54:02.242293 15643 tablet.cc:2369] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:02.247160 15820 raft_consensus.cc:482] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:54:02.247552 15792 tablet.cc:2369] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:02.247506 15820 raft_consensus.cc:504] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.248989 15820 leader_election.cc:288] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e6531e461f34217a42d4b5c14e9129c (127.27.115.195:35861), b7f119a0968247d9805e8a9cf256c5f8 (127.27.115.193:44093)
W20260811 18:54:02.250005 15498 tablet.cc:2369] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:02.254827 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.250223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:60449 (local address 127.27.115.193:44093)
0811 18:54:02.250378 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.250385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.250483 (+ 98us) server_negotiation.cc:407] Connection header received
0811 18:54:02.250859 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.250863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.251051 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.251212 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:02.251527 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.252042 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.253461 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.254092 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.254238 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.254300 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.254596 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.254680 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:54:02.254827 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.249568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:55721 (local address 127.27.115.195:35861)
0811 18:54:02.249814 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.249820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.250312 (+ 492us) server_negotiation.cc:407] Connection header received
0811 18:54:02.250411 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.250415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.250487 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.250577 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:02.251007 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.251734 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.253345 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.254057 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.254238 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.254300 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.254596 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.254681 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 18:54:02.254835 15645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.249411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:35861 (local address 127.27.115.194:55721)
0811 18:54:02.249827 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.249859 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.250113 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.250633 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.250636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.250651 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:02.250838 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.250845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.251853 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.251860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.253174 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.253186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.254270 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.254274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.254430 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.254474 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.254558 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.254665 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":83}
I20260811 18:54:02.254845 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.249836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:44093 (local address 127.27.115.194:60449)
0811 18:54:02.250346 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.250364 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.250637 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.251228 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.251231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.251254 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:02.251392 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.251398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.252167 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.252170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.253321 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.253334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.254270 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.254274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.254416 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.254469 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.254558 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.254665 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":7,"thread_start_us":125,"threads_started":1}
I20260811 18:54:02.255713 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0bdbbde36c4ec89696fed74e42cf72" candidate_uuid: "c073820394884aa1a03dfcdc6bebacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f119a0968247d9805e8a9cf256c5f8" is_pre_election: true
I20260811 18:54:02.255944 15742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0bdbbde36c4ec89696fed74e42cf72" candidate_uuid: "c073820394884aa1a03dfcdc6bebacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6531e461f34217a42d4b5c14e9129c" is_pre_election: true
I20260811 18:54:02.256131 15448 raft_consensus.cc:2454] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c073820394884aa1a03dfcdc6bebacd8 in term 0.
I20260811 18:54:02.256196 15742 raft_consensus.cc:2454] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c073820394884aa1a03dfcdc6bebacd8 in term 0.
I20260811 18:54:02.256701 15524 leader_election.cc:302] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6531e461f34217a42d4b5c14e9129c, c073820394884aa1a03dfcdc6bebacd8; no voters:
I20260811 18:54:02.257030 15820 raft_consensus.cc:2781] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:02.257117 15820 raft_consensus.cc:482] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:02.257148 15820 raft_consensus.cc:3037] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:02.258409 15820 raft_consensus.cc:504] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.258910 15820 leader_election.cc:288] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [CANDIDATE]: Term 1 election: Requested vote from peers 2e6531e461f34217a42d4b5c14e9129c (127.27.115.195:35861), b7f119a0968247d9805e8a9cf256c5f8 (127.27.115.193:44093)
I20260811 18:54:02.259452 15742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0bdbbde36c4ec89696fed74e42cf72" candidate_uuid: "c073820394884aa1a03dfcdc6bebacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6531e461f34217a42d4b5c14e9129c"
I20260811 18:54:02.259541 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0bdbbde36c4ec89696fed74e42cf72" candidate_uuid: "c073820394884aa1a03dfcdc6bebacd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f119a0968247d9805e8a9cf256c5f8"
I20260811 18:54:02.259624 15742 raft_consensus.cc:3037] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:02.259706 15448 raft_consensus.cc:3037] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:02.260893 15448 raft_consensus.cc:2454] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c073820394884aa1a03dfcdc6bebacd8 in term 1.
I20260811 18:54:02.260900 15742 raft_consensus.cc:2454] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c073820394884aa1a03dfcdc6bebacd8 in term 1.
I20260811 18:54:02.261355 15524 leader_election.cc:302] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6531e461f34217a42d4b5c14e9129c, c073820394884aa1a03dfcdc6bebacd8; no voters:
I20260811 18:54:02.261619 15820 raft_consensus.cc:2781] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:02.261933 15820 raft_consensus.cc:686] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 1 LEADER]: Becoming Leader. State: Replica: c073820394884aa1a03dfcdc6bebacd8, State: Running, Role: LEADER
I20260811 18:54:02.262393 15820 consensus_queue.cc:237] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } }
I20260811 18:54:02.266644 15289 catalog_manager.cc:5495] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c073820394884aa1a03dfcdc6bebacd8 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "c073820394884aa1a03dfcdc6bebacd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c073820394884aa1a03dfcdc6bebacd8" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 32809 } health_report { overall_health: HEALTHY } } }
I20260811 18:54:02.312721 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.308515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.27.115.194:32809)
0811 18:54:02.308794 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.308800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.308818 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:02.308990 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.308993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.309063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.309193 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:02.309492 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.310030 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.311018 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.311253 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.311378 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.311737 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:02.312293 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:02.312435 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.312479 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.312547 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260811 18:54:02.320035 15448 raft_consensus.cc:1261] T ca0bdbbde36c4ec89696fed74e42cf72 P b7f119a0968247d9805e8a9cf256c5f8 [term 1 FOLLOWER]: Refusing update from remote peer c073820394884aa1a03dfcdc6bebacd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:02.320046 15742 raft_consensus.cc:1261] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 1 FOLLOWER]: Refusing update from remote peer c073820394884aa1a03dfcdc6bebacd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:02.320945 15822 consensus_queue.cc:1035] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e6531e461f34217a42d4b5c14e9129c" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 35861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:02.321184 15820 consensus_queue.cc:1035] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f119a0968247d9805e8a9cf256c5f8" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:02.334893 15826 mvcc.cc:205] Tried to move back new op lower bound from 7317399315734138880 to 7317399315510095872. Current Snapshot: MvccSnapshot[applied={T|T < 7317399315734138880}]
I20260811 18:54:02.339892 15289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:02.340128 15289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:02.343420 15289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ffff12c438c486baea90821a083c647: Sending DeleteTablet for 3 replicas of tablet ca0bdbbde36c4ec89696fed74e42cf72
I20260811 18:54:02.344506 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:50774
I20260811 18:54:02.344627 15721 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:42574
I20260811 18:54:02.345173 15834 tablet_replica.cc:330] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: stopping tablet replica
I20260811 18:54:02.345245 15835 tablet_replica.cc:330] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: stopping tablet replica
I20260811 18:54:02.345211 15425 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0bdbbde36c4ec89696fed74e42cf72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:02 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:54002
I20260811 18:54:02.345667 15835 raft_consensus.cc:2229] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:02.345683 15834 raft_consensus.cc:2229] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:54:02.346060 15835 raft_consensus.cc:2258] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:02.346341 15834 raft_consensus.cc:2258] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:02.346778 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15354
I20260811 18:54:02.348290 15835 ts_tablet_manager.cc:1905] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:02.348591 15834 ts_tablet_manager.cc:1905] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:02.351356 15835 ts_tablet_manager.cc:1918] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:02.351478 15835 log.cc:1196] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/ca0bdbbde36c4ec89696fed74e42cf72
I20260811 18:54:02.351848 15835 ts_tablet_manager.cc:1939] T ca0bdbbde36c4ec89696fed74e42cf72 P 2e6531e461f34217a42d4b5c14e9129c: Deleting consensus metadata
I20260811 18:54:02.352736 15834 ts_tablet_manager.cc:1918] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:02.352948 15834 log.cc:1196] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/ca0bdbbde36c4ec89696fed74e42cf72
I20260811 18:54:02.353371 15274 catalog_manager.cc:4841] TS 2e6531e461f34217a42d4b5c14e9129c (127.27.115.195:35861): tablet ca0bdbbde36c4ec89696fed74e42cf72 (table test-table [id=09e7818e7cf346579a226a99b13c90ff]) successfully deleted
I20260811 18:54:02.353684 15834 ts_tablet_manager.cc:1939] T ca0bdbbde36c4ec89696fed74e42cf72 P c073820394884aa1a03dfcdc6bebacd8: Deleting consensus metadata
I20260811 18:54:02.355278 15272 catalog_manager.cc:4841] TS c073820394884aa1a03dfcdc6bebacd8 (127.27.115.194:32809): tablet ca0bdbbde36c4ec89696fed74e42cf72 (table test-table [id=09e7818e7cf346579a226a99b13c90ff]) successfully deleted
W20260811 18:54:02.360883 15274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:44093 (error 108)
I20260811 18:54:02.361003 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15502
I20260811 18:54:02.365001 15809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.364487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:44093 (local address 127.0.0.1:54008)
0811 18:54:02.364797 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.364904 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:44093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
W20260811 18:54:02.365239 15274 catalog_manager.cc:4584] TS b7f119a0968247d9805e8a9cf256c5f8 (127.27.115.193:44093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca0bdbbde36c4ec89696fed74e42cf72: Network error: Client connection negotiation failed: client connection to 127.27.115.193:44093: connect: Connection refused (error 111)
I20260811 18:54:02.370575 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15648
I20260811 18:54:02.382442 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15253
2026-08-11T18:54:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2667 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T18:54:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:02Z Disabled control of system clock
I20260811 18:54:02.419960 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33999
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:33999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:02.539484 15841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:02.539806 15841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:02.539844 15841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:02.544296 15841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:02.544382 15841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:02.544399 15841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:02.544415 15841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:02.544431 15841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:02.549590 15841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:33999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:02.550889 15841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:02.552366 15841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:02.561235 15851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:02.561229 15848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:02.561239 15847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:02.561630 15841 server_base.cc:1034] running on GCE node
I20260811 18:54:02.562356 15841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:02.563558 15841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:02.564756 15841 hybrid_clock.cc:648] HybridClock initialized: now 1786474442564725 us; error 44 us; skew 500 ppm
I20260811 18:54:02.566967 15841 webserver.cc:460] Webserver started at http://127.27.115.254:37319/ using document root <none> and password file <none>
I20260811 18:54:02.567610 15841 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:02.567679 15841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:02.567926 15841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:02.570235 15841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "28f97c3ff6f94b37ac396ef855eca032"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "e6379ddc2452c5934f2de71f751155a7"
server_key_iv: "d5ddc2e0bc3298cb7069b44e876c2e5c"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.570842 15841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "28f97c3ff6f94b37ac396ef855eca032"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "e6379ddc2452c5934f2de71f751155a7"
server_key_iv: "d5ddc2e0bc3298cb7069b44e876c2e5c"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.574829 15841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:54:02.577569 15858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:02.579169 15841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:02.579308 15841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "28f97c3ff6f94b37ac396ef855eca032"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "e6379ddc2452c5934f2de71f751155a7"
server_key_iv: "d5ddc2e0bc3298cb7069b44e876c2e5c"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.579414 15841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:02.610821 15841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:02.611559 15841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:02.611728 15841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:02.619021 15841 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:33999
I20260811 18:54:02.619024 15914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:33999 every 8 connection(s)
I20260811 18:54:02.620119 15841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:02.623594 15915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:02.627923 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15841
I20260811 18:54:02.628041 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:02.628342 28111 external_mini_cluster.cc:1428] Setting key cc1db7f60e78efb96507cd355f3b7f8d
I20260811 18:54:02.630079 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: Bootstrap starting.
I20260811 18:54:02.633474 15915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:02.634521 15915 log.cc:824] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:02.637161 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: No bootstrap required, opened a new log
I20260811 18:54:02.640671 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.630058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.27.115.254:33999)
0811 18:54:02.631031 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.631054 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.631099 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:54:02.632189 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.632215 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.632756 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.633168 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:02.633711 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.635203 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.636269 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.636982 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.637163 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.638199 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:02.638207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:02.638224 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:02.638743 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:02.638818 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:02.638846 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:02.638949 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.639116 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.639431 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":194,"threads_started":1}
I20260811 18:54:02.641345 15915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } }
I20260811 18:54:02.641615 15915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:02.641727 15915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f97c3ff6f94b37ac396ef855eca032, State: Initialized, Role: FOLLOWER
I20260811 18:54:02.642304 15915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } }
I20260811 18:54:02.642474 15915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:02.642544 15915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:02.642652 15915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:02.643879 15915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } }
I20260811 18:54:02.644191 15915 leader_election.cc:302] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28f97c3ff6f94b37ac396ef855eca032; no voters:
I20260811 18:54:02.644474 15915 leader_election.cc:288] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:02.644701 15920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:02.644956 15920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [term 1 LEADER]: Becoming Leader. State: Replica: 28f97c3ff6f94b37ac396ef855eca032, State: Running, Role: LEADER
I20260811 18:54:02.645314 15920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } }
I20260811 18:54:02.645812 15915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:02.647135 15922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f97c3ff6f94b37ac396ef855eca032. Latest consensus state: current_term: 1 leader_uuid: "28f97c3ff6f94b37ac396ef855eca032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } } }
I20260811 18:54:02.647166 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28f97c3ff6f94b37ac396ef855eca032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f97c3ff6f94b37ac396ef855eca032" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33999 } } }
I20260811 18:54:02.647284 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:02.647284 15922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:02.647789 15929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:02.651118 15929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:02.657539 15929 catalog_manager.cc:1347] Generated new cluster ID: 49c4c8533a6b4e76bfe6dd6801d3434d
I20260811 18:54:02.657750 15929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:02.693953 15929 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:02.695109 15929 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:02.703159 15929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: Generated new TSK 0
I20260811 18:54:02.704028 15929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:02.711153 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--builtin_ntp_servers=127.27.115.212:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:02.860703 15940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:02.861032 15940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:02.861078 15940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:02.865860 15940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:02.865962 15940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:02.866024 15940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:02.871726 15940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:02.873023 15940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:02.874682 15940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:02.885537 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:02.885493 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:02.885715 15940 server_base.cc:1034] running on GCE node
W20260811 18:54:02.885493 15950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:02.886448 15940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:02.887192 15940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:02.888495 15940 hybrid_clock.cc:648] HybridClock initialized: now 1786474442888477 us; error 32 us; skew 500 ppm
I20260811 18:54:02.890906 15940 webserver.cc:460] Webserver started at http://127.27.115.193:35461/ using document root <none> and password file <none>
I20260811 18:54:02.891551 15940 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:02.891652 15940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:02.891901 15940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:02.894086 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "a1432c4d250b4068a64a0fb10dfc0154"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "b3994b7ec0492ff8fcf20d84318e4c42"
server_key_iv: "c122c303b11382d5b6fa35093e97ec00"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.894770 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "a1432c4d250b4068a64a0fb10dfc0154"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "b3994b7ec0492ff8fcf20d84318e4c42"
server_key_iv: "c122c303b11382d5b6fa35093e97ec00"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.898893 15940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:02.901823 15957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:02.903312 15940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:02.903481 15940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "a1432c4d250b4068a64a0fb10dfc0154"
format_stamp: "Formatted at 2026-08-11 18:54:02 on dist-test-slave-9jjr"
server_key: "b3994b7ec0492ff8fcf20d84318e4c42"
server_key_iv: "c122c303b11382d5b6fa35093e97ec00"
server_key_version: "encryptionkey@0"
I20260811 18:54:02.903625 15940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:02.930497 15940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:02.931334 15940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:02.931566 15940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:02.932279 15940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:02.933423 15940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:02.933506 15940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:02.933588 15940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:02.933631 15940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:02.944594 15940 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:46327
I20260811 18:54:02.944614 16076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:46327 every 8 connection(s)
I20260811 18:54:02.945722 15940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:02.952085 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 15940
I20260811 18:54:02.952236 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:02.952564 28111 external_mini_cluster.cc:1428] Setting key 99b36154ea6305d2d6d827ae1ba46668
I20260811 18:54:02.955281 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--builtin_ntp_servers=127.27.115.212:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:02.957505 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.948068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:43983 (local address 127.27.115.254:33999)
0811 18:54:02.948216 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:02.948219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:02.949514 (+ 1295us) server_negotiation.cc:407] Connection header received
0811 18:54:02.951019 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:02.951022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:02.951076 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:02.951161 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:02.952437 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.953087 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.954863 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.955305 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.955438 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.956834 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:02.956840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:02.956847 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:02.956867 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:02.956894 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:02.956907 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:02.956997 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:02.957328 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 18:54:02.957387 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:54:02.959151 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:02.948541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33999 (local address 127.27.115.193:43983)
0811 18:54:02.949345 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:02.949392 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:02.950765 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:02.951349 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:02.951356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:02.951446 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:02.952170 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.952197 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.953286 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.953292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.954702 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:02.954716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:02.955560 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:02.955565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:02.955769 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:02.956525 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:02.956527 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:02.956575 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:02.956651 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:02.957038 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:02.957052 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:02.957059 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:54:02.957281 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 18:54:02.957910 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":138,"threads_started":1}
I20260811 18:54:02.960619 16077 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33999
I20260811 18:54:02.960965 16077 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:02.961853 16077 heartbeater.cc:511] Master 127.27.115.254:33999 requested a full tablet report, sending...
I20260811 18:54:02.963868 15875 ts_manager.cc:194] Registered new tserver with Master: a1432c4d250b4068a64a0fb10dfc0154 (127.27.115.193:46327)
I20260811 18:54:02.965142 15875 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.193:43983
W20260811 18:54:03.081818 16083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:03.082139 16083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:03.082269 16083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:03.086720 16083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:03.086810 16083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:03.086899 16083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:03.092074 16083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:03.093396 16083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:03.095072 16083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:03.106415 16089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:03.106585 16083 server_base.cc:1034] running on GCE node
W20260811 18:54:03.106415 16090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:03.106416 16092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:03.107234 16083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:03.108012 16083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:03.109236 16083 hybrid_clock.cc:648] HybridClock initialized: now 1786474443109200 us; error 51 us; skew 500 ppm
I20260811 18:54:03.111524 16083 webserver.cc:460] Webserver started at http://127.27.115.194:42453/ using document root <none> and password file <none>
I20260811 18:54:03.112174 16083 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:03.112283 16083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:03.112537 16083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:03.114878 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "364756d279c04f3e8a88402bd976e9e5"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "270596499e114fc7b7bbff7cba6574fd"
server_key_iv: "caedce7b57e96c8241c90ac9bb5a46a6"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.115515 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "364756d279c04f3e8a88402bd976e9e5"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "270596499e114fc7b7bbff7cba6574fd"
server_key_iv: "caedce7b57e96c8241c90ac9bb5a46a6"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.119601 16083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:03.122401 16099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.123818 16083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:03.123980 16083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "364756d279c04f3e8a88402bd976e9e5"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "270596499e114fc7b7bbff7cba6574fd"
server_key_iv: "caedce7b57e96c8241c90ac9bb5a46a6"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.124116 16083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:03.142750 16083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:03.143661 16083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:03.143880 16083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:03.144577 16083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:03.145749 16083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:03.145823 16083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.145902 16083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:03.145984 16083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.157461 16083 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:45255
I20260811 18:54:03.157501 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:45255 every 8 connection(s)
I20260811 18:54:03.158664 16083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:03.164623 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16083
I20260811 18:54:03.164785 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:03.165148 28111 external_mini_cluster.cc:1428] Setting key 0d2fbc63b43b65ed9d91d556904f5ed7
I20260811 18:54:03.167737 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--builtin_ntp_servers=127.27.115.212:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:03.171078 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.161121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:58613 (local address 127.27.115.254:33999)
0811 18:54:03.161293 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.161297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.162674 (+ 1377us) server_negotiation.cc:407] Connection header received
0811 18:54:03.164369 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.164373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.164449 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.164554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:03.165889 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.166624 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.168265 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.168670 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.168837 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.170522 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:03.170526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:03.170528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:03.170545 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:03.170569 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:03.170578 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:03.170641 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.170874 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.170944 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:54:03.172586 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.161449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33999 (local address 127.27.115.194:58613)
0811 18:54:03.162370 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.162517 (+ 147us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.164071 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.164805 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.164816 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.164980 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:03.165671 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.165694 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.166885 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.166890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.168101 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.168115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.168907 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.168911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.169153 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.170037 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:03.170041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:03.170117 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:03.170269 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:03.170678 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:03.170693 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:03.170701 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.170867 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.171276 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":152,"threads_started":1}
I20260811 18:54:03.174512 16221 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33999
I20260811 18:54:03.174846 16221 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:03.175513 16221 heartbeater.cc:511] Master 127.27.115.254:33999 requested a full tablet report, sending...
I20260811 18:54:03.177008 15875 ts_manager.cc:194] Registered new tserver with Master: 364756d279c04f3e8a88402bd976e9e5 (127.27.115.194:45255)
I20260811 18:54:03.177975 15875 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.194:58613
W20260811 18:54:03.289372 16224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:03.289681 16224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:03.289721 16224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:03.293865 16224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:03.293941 16224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:03.293998 16224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:03.298916 16224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:33999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:03.300091 16224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:03.301662 16224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:03.312762 16233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:03.312896 16229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:03.312925 16224 server_base.cc:1034] running on GCE node
W20260811 18:54:03.312849 16230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:03.313475 16224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:03.314265 16224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:03.315434 16224 hybrid_clock.cc:648] HybridClock initialized: now 1786474443315415 us; error 31 us; skew 500 ppm
I20260811 18:54:03.317348 16224 webserver.cc:460] Webserver started at http://127.27.115.195:36593/ using document root <none> and password file <none>
I20260811 18:54:03.317968 16224 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:03.318032 16224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:03.318315 16224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:03.320497 16224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "1d647a2c486757f5f352cf654f2c776c"
server_key_iv: "1da6ecf901a832f7493d684f3efc7b14"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.321116 16224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "1d647a2c486757f5f352cf654f2c776c"
server_key_iv: "1da6ecf901a832f7493d684f3efc7b14"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.325253 16224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:54:03.327957 16241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.329421 16224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:03.329558 16224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78"
format_stamp: "Formatted at 2026-08-11 18:54:03 on dist-test-slave-9jjr"
server_key: "1d647a2c486757f5f352cf654f2c776c"
server_key_iv: "1da6ecf901a832f7493d684f3efc7b14"
server_key_version: "encryptionkey@0"
I20260811 18:54:03.329695 16224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:03.361250 16224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:03.362044 16224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:03.362293 16224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:03.362962 16224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:03.364051 16224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:03.364126 16224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.364200 16224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:03.364284 16224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:03.374342 16224 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:42319
I20260811 18:54:03.374372 16364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:42319 every 8 connection(s)
I20260811 18:54:03.375355 16224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:03.378036 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16224
I20260811 18:54:03.378216 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:03.378566 28111 external_mini_cluster.cc:1428] Setting key 374e5006624d7ddfd978e54f65065d46
I20260811 18:54:03.387557 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.377381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:45981 (local address 127.27.115.254:33999)
0811 18:54:03.377529 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.377533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.378935 (+ 1402us) server_negotiation.cc:407] Connection header received
0811 18:54:03.380986 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.380990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.381048 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.381137 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:03.382675 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.383388 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.384783 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.385024 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.385117 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.386783 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:03.386787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:03.386790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:03.386808 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:03.386833 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:03.386841 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:03.386917 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.387333 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.387418 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:03.388744 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.377801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:33999 (local address 127.27.115.195:45981)
0811 18:54:03.378745 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.378810 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.380750 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.381503 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.381512 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.381621 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:03.382486 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.382510 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.383535 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.383538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.384623 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.384636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.385231 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.385234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.385427 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.386416 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:03.386418 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:03.386470 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:03.386576 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:03.386947 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:03.386957 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:03.386962 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.387263 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.387596 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":135,"threads_started":1}
I20260811 18:54:03.390599 16365 heartbeater.cc:347] Connected to a master server at 127.27.115.254:33999
I20260811 18:54:03.390957 16365 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:03.391738 16365 heartbeater.cc:511] Master 127.27.115.254:33999 requested a full tablet report, sending...
I20260811 18:54:03.393217 15875 ts_manager.cc:194] Registered new tserver with Master: 7210bc4ba2a348a2aa8f78fbda5c9b78 (127.27.115.195:42319)
I20260811 18:54:03.394066 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:03.394302 15875 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.195:45981
I20260811 18:54:03.401428 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.396706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.27.115.254:33999)
0811 18:54:03.396853 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.396860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.397064 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:54:03.397234 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.397240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.397301 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.397403 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:03.397751 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.398336 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.399326 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.399566 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.399694 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.400806 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:03.400810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:03.400813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:03.400840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:03.400864 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:03.400874 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:03.401014 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.401161 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.401245 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:03.405443 15875 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:03.409631 15875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:03.426759 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.27.115.194:45255)
0811 18:54:03.421439 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.421452 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.421751 (+ 299us) server_negotiation.cc:407] Connection header received
0811 18:54:03.421836 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.421840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.422029 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.422245 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:03.422822 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.423707 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.425307 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.425920 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.426081 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.426341 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.426460 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.426570 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":144,"threads_started":1}
I20260811 18:54:03.426754 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45255 (local address 127.0.0.1:49292)
0811 18:54:03.421636 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.421648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.421745 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.422399 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.422406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.422436 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:03.422669 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.422681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.423863 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.423867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.425153 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.425164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.426074 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.426078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.426236 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.426282 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.426422 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.426570 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":251,"spinlock_wait_cycles":1920,"thread_start_us":97,"threads_started":1}
I20260811 18:54:03.427246 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.27.115.195:42319)
0811 18:54:03.420855 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.420869 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.421578 (+ 709us) server_negotiation.cc:407] Connection header received
0811 18:54:03.421854 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.421865 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.422030 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.422211 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:03.422839 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.424100 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.425373 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.426246 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.426371 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.426552 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.426903 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.427036 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":155,"threads_started":1}
I20260811 18:54:03.427698 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:42319 (local address 127.0.0.1:40536)
0811 18:54:03.421456 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.421472 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.421665 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.422433 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.422436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.422448 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:03.422662 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.422670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.424266 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.424270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.425196 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.425208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.426615 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.426618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.426714 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.426738 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.427479 (+ 741us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.427564 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":703,"thread_start_us":103,"threads_started":1}
I20260811 18:54:03.429484 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46327 (local address 127.0.0.1:53970)
0811 18:54:03.421238 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.421307 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.421489 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.422567 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.422570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.422586 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:03.422706 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.422715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.424266 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.424270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.426404 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.426425 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.428812 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.428815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.428900 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.428919 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.429020 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.429324 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":909,"thread_start_us":119,"threads_started":1}
I20260811 18:54:03.429541 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.420229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53970 (local address 127.27.115.193:46327)
0811 18:54:03.421316 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.421323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.421664 (+ 341us) server_negotiation.cc:407] Connection header received
0811 18:54:03.421780 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.421786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.422030 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.422245 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:03.422869 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.424100 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.426586 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.428379 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.428552 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.428751 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.429217 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.429363 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":957,"thread_start_us":122,"threads_started":1}
I20260811 18:54:03.430934 16293 tablet_service.cc:1467] Processing CreateTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf (DEFAULT_TABLE table=test-table [id=41a56bd83fd7499eba4c8ed72fc6925c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:03.430934 16150 tablet_service.cc:1467] Processing CreateTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf (DEFAULT_TABLE table=test-table [id=41a56bd83fd7499eba4c8ed72fc6925c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:03.432869 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f70dac3cce1e4eca872cff01fe5bb6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:03.432894 16293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f70dac3cce1e4eca872cff01fe5bb6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:03.434038 16010 tablet_service.cc:1467] Processing CreateTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf (DEFAULT_TABLE table=test-table [id=41a56bd83fd7499eba4c8ed72fc6925c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:03.435509 16010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f70dac3cce1e4eca872cff01fe5bb6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:03.442739 16386 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Bootstrap starting.
I20260811 18:54:03.443338 16385 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Bootstrap starting.
I20260811 18:54:03.445236 16387 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: Bootstrap starting.
I20260811 18:54:03.446419 16385 tablet_bootstrap.cc:654] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:03.446769 16386 tablet_bootstrap.cc:654] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:03.447463 16385 log.cc:824] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:03.447896 16387 tablet_bootstrap.cc:654] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:03.447891 16386 log.cc:824] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:03.448726 16387 log.cc:824] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:03.449880 16385 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: No bootstrap required, opened a new log
I20260811 18:54:03.450176 16385 ts_tablet_manager.cc:1397] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:54:03.451330 16387 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: No bootstrap required, opened a new log
I20260811 18:54:03.451330 16386 tablet_bootstrap.cc:492] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: No bootstrap required, opened a new log
I20260811 18:54:03.451550 16386 ts_tablet_manager.cc:1397] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 18:54:03.451550 16387 ts_tablet_manager.cc:1397] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:54:03.454090 16385 raft_consensus.cc:348] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.454478 16385 raft_consensus.cc:374] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:03.454545 16385 raft_consensus.cc:729] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7210bc4ba2a348a2aa8f78fbda5c9b78, State: Initialized, Role: FOLLOWER
I20260811 18:54:03.455065 16385 consensus_queue.cc:260] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.455960 16365 heartbeater.cc:503] Master 127.27.115.254:33999 was elected leader, sending a full tablet report...
I20260811 18:54:03.456838 16386 raft_consensus.cc:348] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.456835 16387 raft_consensus.cc:348] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.457176 16387 raft_consensus.cc:374] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:03.457176 16386 raft_consensus.cc:374] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:03.457233 16386 raft_consensus.cc:729] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 364756d279c04f3e8a88402bd976e9e5, State: Initialized, Role: FOLLOWER
I20260811 18:54:03.457233 16387 raft_consensus.cc:729] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1432c4d250b4068a64a0fb10dfc0154, State: Initialized, Role: FOLLOWER
I20260811 18:54:03.457803 16387 consensus_queue.cc:260] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.457803 16386 consensus_queue.cc:260] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.458925 16221 heartbeater.cc:503] Master 127.27.115.254:33999 was elected leader, sending a full tablet report...
I20260811 18:54:03.459066 16386 ts_tablet_manager.cc:1428] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 18:54:03.459066 16387 ts_tablet_manager.cc:1428] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 18:54:03.459651 16077 heartbeater.cc:503] Master 127.27.115.254:33999 was elected leader, sending a full tablet report...
I20260811 18:54:03.459753 16385 ts_tablet_manager.cc:1428] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260811 18:54:03.557897 16392 raft_consensus.cc:482] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:54:03.558256 16392 raft_consensus.cc:504] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.559695 16392 leader_election.cc:288] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1432c4d250b4068a64a0fb10dfc0154 (127.27.115.193:46327), 364756d279c04f3e8a88402bd976e9e5 (127.27.115.194:45255)
I20260811 18:54:03.563694 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.560057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46327 (local address 127.27.115.195:38851)
0811 18:54:03.560224 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.560243 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.560365 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.560691 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.560694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.560713 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:03.560850 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.560856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.561618 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.561621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.562527 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.562538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.563237 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.563241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.563351 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.563369 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.563465 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.563546 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:54:03.563694 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.560164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:38851 (local address 127.27.115.193:46327)
0811 18:54:03.560307 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.560312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.560328 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:54:03.560498 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.560501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.560562 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.560658 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:03.560968 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.561489 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.562659 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.563090 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.563237 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.563295 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.563485 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.563561 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:54:03.564101 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.560286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:49501 (local address 127.27.115.194:45255)
0811 18:54:03.560438 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.560444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.560834 (+ 390us) server_negotiation.cc:407] Connection header received
0811 18:54:03.561011 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.561014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.561068 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.561152 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:03.561475 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.561973 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.563071 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.563467 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.563599 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.563649 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.563885 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.563981 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:54:03.564504 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.560057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:45255 (local address 127.27.115.195:49501)
0811 18:54:03.560715 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.560734 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:03.560883 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:03.561166 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:03.561169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:03.561187 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:03.561345 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.561350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.562095 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.562100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.562921 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:03.562931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.563594 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.563598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:03.563701 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.563723 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:54:03.564279 (+ 556us) client_negotiation.cc:240] Negotiation successful
0811 18:54:03.564358 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":300,"thread_start_us":168,"threads_started":1}
I20260811 18:54:03.564494 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70dac3cce1e4eca872cff01fe5bb6bf" candidate_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1432c4d250b4068a64a0fb10dfc0154" is_pre_election: true
I20260811 18:54:03.564864 16031 raft_consensus.cc:2454] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7210bc4ba2a348a2aa8f78fbda5c9b78 in term 0.
I20260811 18:54:03.565294 16173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70dac3cce1e4eca872cff01fe5bb6bf" candidate_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "364756d279c04f3e8a88402bd976e9e5" is_pre_election: true
I20260811 18:54:03.565492 16245 leader_election.cc:302] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7210bc4ba2a348a2aa8f78fbda5c9b78, a1432c4d250b4068a64a0fb10dfc0154; no voters:
I20260811 18:54:03.565601 16173 raft_consensus.cc:2454] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7210bc4ba2a348a2aa8f78fbda5c9b78 in term 0.
I20260811 18:54:03.565838 16392 raft_consensus.cc:2781] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:03.565960 16392 raft_consensus.cc:482] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:03.566038 16392 raft_consensus.cc:3037] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:03.567304 16392 raft_consensus.cc:504] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.567759 16392 leader_election.cc:288] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [CANDIDATE]: Term 1 election: Requested vote from peers a1432c4d250b4068a64a0fb10dfc0154 (127.27.115.193:46327), 364756d279c04f3e8a88402bd976e9e5 (127.27.115.194:45255)
I20260811 18:54:03.568671 16173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70dac3cce1e4eca872cff01fe5bb6bf" candidate_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "364756d279c04f3e8a88402bd976e9e5"
I20260811 18:54:03.568877 16173 raft_consensus.cc:3037] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:03.570104 16173 raft_consensus.cc:2454] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7210bc4ba2a348a2aa8f78fbda5c9b78 in term 1.
I20260811 18:54:03.570408 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70dac3cce1e4eca872cff01fe5bb6bf" candidate_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1432c4d250b4068a64a0fb10dfc0154"
I20260811 18:54:03.570618 16031 raft_consensus.cc:3037] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:03.570775 16244 leader_election.cc:302] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 364756d279c04f3e8a88402bd976e9e5, 7210bc4ba2a348a2aa8f78fbda5c9b78; no voters:
I20260811 18:54:03.571033 16392 raft_consensus.cc:2781] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:03.571331 16392 raft_consensus.cc:686] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 1 LEADER]: Becoming Leader. State: Replica: 7210bc4ba2a348a2aa8f78fbda5c9b78, State: Running, Role: LEADER
I20260811 18:54:03.571791 16392 consensus_queue.cc:237] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } }
I20260811 18:54:03.572353 16031 raft_consensus.cc:2454] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7210bc4ba2a348a2aa8f78fbda5c9b78 in term 1.
I20260811 18:54:03.576035 15873 catalog_manager.cc:5495] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7210bc4ba2a348a2aa8f78fbda5c9b78 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7210bc4ba2a348a2aa8f78fbda5c9b78" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 42319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 } health_report { overall_health: UNKNOWN } } }
I20260811 18:54:03.622555 16380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.618633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.27.115.195:42319)
0811 18:54:03.618799 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:03.618804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:03.618822 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:03.618924 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:03.618927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:03.618983 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:03.619103 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:03.619370 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.619886 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.620850 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:03.621072 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:03.621196 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:03.621527 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:03.622054 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:03.622231 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:03.622296 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 18:54:03.622357 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:54:03.626374 16367 tablet.cc:2369] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:03.629949 16031 raft_consensus.cc:1261] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 1 FOLLOWER]: Refusing update from remote peer 7210bc4ba2a348a2aa8f78fbda5c9b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:03.630038 16173 raft_consensus.cc:1261] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 1 FOLLOWER]: Refusing update from remote peer 7210bc4ba2a348a2aa8f78fbda5c9b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:03.630798 16397 consensus_queue.cc:1035] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1432c4d250b4068a64a0fb10dfc0154" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 46327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:03.631024 16392 consensus_queue.cc:1035] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "364756d279c04f3e8a88402bd976e9e5" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:03.639029 16400 mvcc.cc:205] Tried to move back new op lower bound from 7317399321099120640 to 7317399320874213376. Current Snapshot: MvccSnapshot[applied={T|T < 7317399321099120640}]
I20260811 18:54:03.641547 16404 mvcc.cc:205] Tried to move back new op lower bound from 7317399321099120640 to 7317399320874213376. Current Snapshot: MvccSnapshot[applied={T|T < 7317399321099120640}]
I20260811 18:54:03.641152 16401 mvcc.cc:205] Tried to move back new op lower bound from 7317399321099120640 to 7317399320874213376. Current Snapshot: MvccSnapshot[applied={T|T < 7317399321099120640}]
I20260811 18:54:03.651989 15875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:03.652251 15875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:03.655463 15875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28f97c3ff6f94b37ac396ef855eca032: Sending DeleteTablet for 3 replicas of tablet f70dac3cce1e4eca872cff01fe5bb6bf
I20260811 18:54:03.656800 16293 tablet_service.cc:1514] Processing DeleteTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:03 UTC) from {username='slave'} at 127.0.0.1:40536
I20260811 18:54:03.656800 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:03 UTC) from {username='slave'} at 127.0.0.1:49292
I20260811 18:54:03.657037 16010 tablet_service.cc:1514] Processing DeleteTablet for tablet f70dac3cce1e4eca872cff01fe5bb6bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:03 UTC) from {username='slave'} at 127.0.0.1:53970
I20260811 18:54:03.657531 16412 tablet_replica.cc:330] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: stopping tablet replica
I20260811 18:54:03.657830 16413 tablet_replica.cc:330] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154: stopping tablet replica
I20260811 18:54:03.657979 16412 raft_consensus.cc:2229] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:03.658159 16413 raft_consensus.cc:2229] T f70dac3cce1e4eca872cff01fe5bb6bf P a1432c4d250b4068a64a0fb10dfc0154 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:03.658443 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15940
I20260811 18:54:03.661289 16412 raft_consensus.cc:2258] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:03.663903 16410 tablet_replica.cc:330] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: stopping tablet replica
I20260811 18:54:03.664108 16412 ts_tablet_manager.cc:1905] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:03.664410 16410 raft_consensus.cc:2229] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:54:03.665011 16410 raft_consensus.cc:2258] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:03.667115 16410 ts_tablet_manager.cc:1905] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:03.668016 16412 ts_tablet_manager.cc:1918] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:03.668159 16412 log.cc:1196] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/f70dac3cce1e4eca872cff01fe5bb6bf
I20260811 18:54:03.668533 16412 ts_tablet_manager.cc:1939] T f70dac3cce1e4eca872cff01fe5bb6bf P 364756d279c04f3e8a88402bd976e9e5: Deleting consensus metadata
I20260811 18:54:03.670380 15861 catalog_manager.cc:4841] TS 364756d279c04f3e8a88402bd976e9e5 (127.27.115.194:45255): tablet f70dac3cce1e4eca872cff01fe5bb6bf (table test-table [id=41a56bd83fd7499eba4c8ed72fc6925c]) successfully deleted
I20260811 18:54:03.670697 16410 ts_tablet_manager.cc:1918] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:03.670825 16410 log.cc:1196] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/f70dac3cce1e4eca872cff01fe5bb6bf
I20260811 18:54:03.671205 16410 ts_tablet_manager.cc:1939] T f70dac3cce1e4eca872cff01fe5bb6bf P 7210bc4ba2a348a2aa8f78fbda5c9b78: Deleting consensus metadata
W20260811 18:54:03.671625 15862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:46327 (error 108)
I20260811 18:54:03.671916 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16083
I20260811 18:54:03.673022 15860 catalog_manager.cc:4841] TS 7210bc4ba2a348a2aa8f78fbda5c9b78 (127.27.115.195:42319): tablet f70dac3cce1e4eca872cff01fe5bb6bf (table test-table [id=41a56bd83fd7499eba4c8ed72fc6925c]) successfully deleted
I20260811 18:54:03.673657 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:03.672574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:46327 (local address 127.0.0.1:53984)
0811 18:54:03.673426 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:03.673527 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:46327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":725}
W20260811 18:54:03.673878 15862 catalog_manager.cc:4584] TS a1432c4d250b4068a64a0fb10dfc0154 (127.27.115.193:46327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f70dac3cce1e4eca872cff01fe5bb6bf: Network error: Client connection negotiation failed: client connection to 127.27.115.193:46327: connect: Connection refused (error 111)
I20260811 18:54:03.684827 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16224
I20260811 18:54:03.694103 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 15841
2026-08-11T18:54:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1311 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:03 dist-test-slave-9jjr krb5kdc[16417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:03 dist-test-slave-9jjr krb5kdc[16417](info): set up 2 sockets
Aug 11 18:54:03 dist-test-slave-9jjr krb5kdc[16417](info): commencing operation
krb5kdc: starting...
W20260811 18:54:04.831120 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:04 dist-test-slave-9jjr krb5kdc[16417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:04.979182 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37621
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:05.101490 16433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:05.101854 16433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:05.101930 16433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:05.106206 16433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:05.106333 16433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:05.106369 16433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:05.106411 16433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:05.106463 16433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:05.112452 16433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:05.113714 16433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:05.114874 16433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:05.124420 16440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.124428 16439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.124481 16442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:05.124724 16433 server_base.cc:1034] running on GCE node
I20260811 18:54:05.125545 16433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:05.126967 16433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:05.128201 16433 hybrid_clock.cc:648] HybridClock initialized: now 1786474445128166 us; error 52 us; skew 500 ppm
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:05.135654 16433 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:05.136794 16433 webserver.cc:460] Webserver started at http://127.27.115.254:35235/ using document root <none> and password file <none>
I20260811 18:54:05.137444 16433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:05.137544 16433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:05.137797 16433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:05.140077 16433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "2cbece0b3a2743beb82d1f40af0e57b0"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "9db549b7406adf30781b57161a0dfdf0"
server_key_iv: "93320975fbcd50fe84e128e8ee9649f4"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.140852 16433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "2cbece0b3a2743beb82d1f40af0e57b0"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "9db549b7406adf30781b57161a0dfdf0"
server_key_iv: "93320975fbcd50fe84e128e8ee9649f4"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.145182 16433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:54:05.148036 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.149561 16433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:05.149760 16433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "2cbece0b3a2743beb82d1f40af0e57b0"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "9db549b7406adf30781b57161a0dfdf0"
server_key_iv: "93320975fbcd50fe84e128e8ee9649f4"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.149919 16433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:05.172079 16433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:05.175501 16433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:05.175745 16433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:05.183105 16433 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37621
I20260811 18:54:05.183117 16508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37621 every 8 connection(s)
I20260811 18:54:05.184144 16433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:05.187075 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:05.188263 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16433
I20260811 18:54:05.188417 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:05.188748 28111 external_mini_cluster.cc:1428] Setting key b79f639d6a40f51a52317d3c3027d7da
I20260811 18:54:05.194031 16509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: Bootstrap starting.
I20260811 18:54:05.196864 16509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:05.197814 16509 log.cc:824] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:05.200563 16509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: No bootstrap required, opened a new log
I20260811 18:54:05.204471 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:05.190218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.27.115.254:37621)
0811 18:54:05.190688 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:05.190704 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:05.190740 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:54:05.191387 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:05.191405 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:05.191736 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:05.192033 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:05.193058 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.194105 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.195252 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.195776 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.195975 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:05.199176 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:05.199182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:05.199200 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:05.199229 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:05.201099 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.201669 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.201676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.201682 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.201819 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.202239 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.202243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.202244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.202709 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:05.202882 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:05.203274 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 18:54:05.203612 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":149,"threads_started":1}
I20260811 18:54:05.205066 16509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } }
I20260811 18:54:05.205348 16509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:05.205430 16509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cbece0b3a2743beb82d1f40af0e57b0, State: Initialized, Role: FOLLOWER
I20260811 18:54:05.205981 16509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } }
I20260811 18:54:05.206178 16509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:05.206251 16509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:05.206353 16509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:05.207496 16509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } }
I20260811 18:54:05.207818 16509 leader_election.cc:302] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cbece0b3a2743beb82d1f40af0e57b0; no voters:
I20260811 18:54:05.208120 16509 leader_election.cc:288] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:05.208343 16514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:05.208698 16514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [term 1 LEADER]: Becoming Leader. State: Replica: 2cbece0b3a2743beb82d1f40af0e57b0, State: Running, Role: LEADER
I20260811 18:54:05.209038 16514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } }
I20260811 18:54:05.209473 16509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:05.210848 16515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } } }
I20260811 18:54:05.211040 16515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:05.211470 16516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cbece0b3a2743beb82d1f40af0e57b0. Latest consensus state: current_term: 1 leader_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbece0b3a2743beb82d1f40af0e57b0" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37621 } } }
I20260811 18:54:05.211577 16516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:05.211642 16524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:05.214829 16524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:05.221493 16524 catalog_manager.cc:1347] Generated new cluster ID: c5cd4c632af842369de06d45e8e7d7bd
I20260811 18:54:05.221622 16524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:05.229508 16524 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:05.230831 16524 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:05.238574 16524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: Generated new TSK 0
I20260811 18:54:05.239442 16524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:54:05.277944 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:05.401382 16535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:05.401679 16535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:05.401724 16535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:05.406018 16535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:05.406093 16535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:05.406230 16535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:05.412132 16535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:05.413398 16535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:05.414556 16535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:05.425287 16540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.425287 16543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.425359 16541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:05.425930 16535 server_base.cc:1034] running on GCE node
I20260811 18:54:05.426492 16535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:05.427197 16535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:05.428452 16535 hybrid_clock.cc:648] HybridClock initialized: now 1786474445428429 us; error 55 us; skew 500 ppm
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:05.431784 16535 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:54:05.432806 16535 webserver.cc:460] Webserver started at http://127.27.115.193:45373/ using document root <none> and password file <none>
I20260811 18:54:05.433509 16535 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:05.433616 16535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:05.433887 16535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:05.436347 16535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "560909cdc81b4d90832e2062f6ab0f4b"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "3234388a034e54fca86be52d8f9fa7ea"
server_key_iv: "62a06cafe0576c837228569351fa6a67"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.437063 16535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "560909cdc81b4d90832e2062f6ab0f4b"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "3234388a034e54fca86be52d8f9fa7ea"
server_key_iv: "62a06cafe0576c837228569351fa6a67"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.441788 16535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:54:05.444988 16551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.446604 16535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:05.446774 16535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "560909cdc81b4d90832e2062f6ab0f4b"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "3234388a034e54fca86be52d8f9fa7ea"
server_key_iv: "62a06cafe0576c837228569351fa6a67"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.446931 16535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:05.477485 16535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:05.481005 16535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:05.481252 16535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:05.481966 16535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:05.482970 16535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:05.483050 16535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.483145 16535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:05.483188 16535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.494789 16535 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:41789
I20260811 18:54:05.494822 16678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:41789 every 8 connection(s)
I20260811 18:54:05.495759 16535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:05.496675 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16535
I20260811 18:54:05.496836 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:05.497185 28111 external_mini_cluster.cc:1428] Setting key 181e12a029647ed68241cf07a5b58dc0
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:54:05.511438 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:05.498061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:33915 (local address 127.27.115.254:37621)
0811 18:54:05.498239 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:05.498242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:05.499281 (+ 1039us) server_negotiation.cc:407] Connection header received
0811 18:54:05.500676 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:05.500679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:05.500739 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:05.500823 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:05.502062 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.502682 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.504005 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.504259 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.504379 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:05.507174 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:05.507180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:05.507186 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:05.507226 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:05.508804 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.509395 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.509397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.509399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.509477 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.509890 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.509894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.509896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.510231 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:05.510348 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:05.511172 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 18:54:05.511313 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:54:05.512789 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:05.498569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37621 (local address 127.27.115.193:33915)
0811 18:54:05.499114 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:05.499163 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:05.500453 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:05.501004 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:05.501011 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:05.501390 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:05.501898 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:05.501912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.502825 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.502833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:05.503875 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:05.503886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.504433 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.504438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:05.504616 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:05.505281 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:05.505299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:05.506971 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:05.508939 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:05.508944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:05.508960 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:05.509250 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:05.509591 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:05.509593 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:05.509594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:05.509763 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:05.510422 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:05.510430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:05.510882 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 18:54:05.511124 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 18:54:05.511555 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":133,"threads_started":1}
I20260811 18:54:05.514261 16679 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37621
I20260811 18:54:05.514532 16679 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:05.515067 16679 heartbeater.cc:511] Master 127.27.115.254:37621 requested a full tablet report, sending...
I20260811 18:54:05.516943 16467 ts_manager.cc:194] Registered new tserver with Master: 560909cdc81b4d90832e2062f6ab0f4b (127.27.115.193:41789)
I20260811 18:54:05.518850 16467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:33915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:54:05.534956 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:05.655895 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:05.656209 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:05.656256 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:05.660784 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:05.660872 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:05.660935 16685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:05.666641 16685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:05.668290 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:05.669505 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:05.680088 16691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.680088 16693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.680299 16690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:05.680816 16685 server_base.cc:1034] running on GCE node
I20260811 18:54:05.681370 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:05.682237 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:05.683435 16685 hybrid_clock.cc:648] HybridClock initialized: now 1786474445683404 us; error 49 us; skew 500 ppm
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:05.686554 16685 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:54:05.687624 16685 webserver.cc:460] Webserver started at http://127.27.115.194:36161/ using document root <none> and password file <none>
I20260811 18:54:05.688257 16685 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:05.688325 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:05.688591 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:05.690934 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "53243c233e204d93940de5fd97f49caf"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "e090cff79b255ac40ce2a6afef24d8a2"
server_key_iv: "fecdafac8323ced77efe296bcce040ad"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.691548 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "53243c233e204d93940de5fd97f49caf"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "e090cff79b255ac40ce2a6afef24d8a2"
server_key_iv: "fecdafac8323ced77efe296bcce040ad"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.697746 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:54:05.701072 16700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.702782 16685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:54:05.702947 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "53243c233e204d93940de5fd97f49caf"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "e090cff79b255ac40ce2a6afef24d8a2"
server_key_iv: "fecdafac8323ced77efe296bcce040ad"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.703107 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:05.742789 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:05.746438 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:05.746662 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:05.747543 16685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:05.748549 16685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:05.748626 16685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.748705 16685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:05.748735 16685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.758546 16685 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:42339
I20260811 18:54:05.758623 16823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:42339 every 8 connection(s)
I20260811 18:54:05.759526 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:05.764304 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16685
I20260811 18:54:05.764489 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:05.764820 28111 external_mini_cluster.cc:1428] Setting key cabae5ddb10f70ee26c88c85c50ef288
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:54:05.777341 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:05.761631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:39481 (local address 127.27.115.254:37621)
0811 18:54:05.761798 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:05.761802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:05.762633 (+ 831us) server_negotiation.cc:407] Connection header received
0811 18:54:05.764087 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:05.764091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:05.764166 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:05.764273 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:05.765900 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.766657 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.767979 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.768486 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.768624 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:05.772523 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:05.772529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:05.772532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:05.772578 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:05.774672 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.775351 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.775357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.775360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.775479 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:05.775896 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:05.775902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:05.775905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:05.776197 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:05.776331 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:05.776964 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 18:54:05.777127 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:54:05.778348 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:05.761938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37621 (local address 127.27.115.194:39481)
0811 18:54:05.762469 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:05.762514 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:05.763855 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:05.764489 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:05.764498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:05.764988 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:05.765688 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:05.765712 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.766816 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.766819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:05.767794 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:05.767805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:05.768761 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:05.768789 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:05.769081 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:05.770030 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:05.770059 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:05.772258 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:05.774850 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:05.774857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:05.774874 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:05.775214 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:05.775594 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:05.775596 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:05.775598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:05.775781 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:05.776379 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:05.776385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:05.776677 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 18:54:05.776866 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:54:05.777197 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":168,"threads_started":1}
I20260811 18:54:05.780273 16824 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37621
I20260811 18:54:05.780592 16824 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:54:05.781296 16824 heartbeater.cc:511] Master 127.27.115.254:37621 requested a full tablet report, sending...
I20260811 18:54:05.782660 16467 ts_manager.cc:194] Registered new tserver with Master: 53243c233e204d93940de5fd97f49caf (127.27.115.194:42339)
I20260811 18:54:05.783588 16467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:39481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:54:05.803920 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:05.928247 16829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:05.928663 16829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:05.928747 16829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:05.933182 16829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:05.933306 16829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:05.933408 16829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:05.939677 16829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:37621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:05.941061 16829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:05.942409 16829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:05.953330 16837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.953325 16835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:05.953329 16834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:05.953639 16829 server_base.cc:1034] running on GCE node
I20260811 18:54:05.954101 16829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:05.954898 16829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:05.956094 16829 hybrid_clock.cc:648] HybridClock initialized: now 1786474445956063 us; error 43 us; skew 500 ppm
Aug 11 18:54:05 dist-test-slave-9jjr krb5kdc[16417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:05.959045 16829 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:54:05.960011 16829 webserver.cc:460] Webserver started at http://127.27.115.195:34877/ using document root <none> and password file <none>
I20260811 18:54:05.960832 16829 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:05.960968 16829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:05.961227 16829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:05.963572 16829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "906c3c693af242b79d2db127b90db323"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "48d92f66a1b92556c2920ac75b2935d7"
server_key_iv: "a31c5de4642b528a5c28a22c37c82ba8"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.964193 16829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "906c3c693af242b79d2db127b90db323"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "48d92f66a1b92556c2920ac75b2935d7"
server_key_iv: "a31c5de4642b528a5c28a22c37c82ba8"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.968219 16829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 18:54:05.970970 16845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.972381 16829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:54:05.972536 16829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "906c3c693af242b79d2db127b90db323"
format_stamp: "Formatted at 2026-08-11 18:54:05 on dist-test-slave-9jjr"
server_key: "48d92f66a1b92556c2920ac75b2935d7"
server_key_iv: "a31c5de4642b528a5c28a22c37c82ba8"
server_key_version: "encryptionkey@0"
I20260811 18:54:05.972664 16829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:05.993754 16829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:05.997167 16829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:05.997438 16829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:05.998101 16829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:05.999110 16829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:05.999185 16829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:05.999261 16829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:05.999311 16829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:06.009338 16829 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:34409
I20260811 18:54:06.009361 16972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:34409 every 8 connection(s)
I20260811 18:54:06.010545 16829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:06.012910 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 16829
I20260811 18:54:06.013077 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:06.013438 28111 external_mini_cluster.cc:1428] Setting key 62f3054c8b930f7ce8b820ed71031ffd
Aug 11 18:54:06 dist-test-slave-9jjr krb5kdc[16417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:06.026974 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.012742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:46875 (local address 127.27.115.254:37621)
0811 18:54:06.012927 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.012932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.014354 (+ 1422us) server_negotiation.cc:407] Connection header received
0811 18:54:06.015839 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.015843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.015899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.015974 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:06.017182 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.017708 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.019017 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.019268 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.019377 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.022616 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:06.022620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:06.022622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:06.022651 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:06.024377 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:06.024993 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:06.024997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:06.024998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:06.025099 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:06.025520 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:06.025523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:06.025524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:06.025772 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:06.025861 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.026612 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.026792 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:54:06.028183 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.013248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:37621 (local address 127.27.115.195:46875)
0811 18:54:06.014013 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.014078 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.015576 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.016124 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.016131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.016515 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:06.017034 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.017047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.017853 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.017856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.018875 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.018886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.019511 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.019514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.019722 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.020553 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:06.020572 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:06.022408 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:06.024514 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:06.024520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:06.024536 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:06.024841 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:06.025215 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:06.025217 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:06.025219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:06.025372 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:06.025886 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:06.025892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:06.026257 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.026505 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.026912 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":156,"threads_started":1}
I20260811 18:54:06.029721 16973 heartbeater.cc:347] Connected to a master server at 127.27.115.254:37621
I20260811 18:54:06.030041 16973 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:06.030711 16973 heartbeater.cc:511] Master 127.27.115.254:37621 requested a full tablet report, sending...
I20260811 18:54:06.031935 16467 ts_manager.cc:194] Registered new tserver with Master: 906c3c693af242b79d2db127b90db323 (127.27.115.195:34409)
I20260811 18:54:06.032627 16467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:46875
I20260811 18:54:06.040683 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:06.052276 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.043209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.27.115.254:37621)
0811 18:54:06.043420 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.043424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.043459 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:54:06.043600 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.043603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.043650 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.043733 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:06.044546 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.045133 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.046226 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.046479 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.046579 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.048305 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:06.048308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:06.048311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:06.048339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:06.049877 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:06.050772 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:06.050776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:06.050778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:06.050876 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:06.051236 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:06.051240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:06.051242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:06.051489 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:06.051682 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.051995 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.052101 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 18:54:06.056095 16467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:06.059305 16467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:06.074765 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.068324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:34409 (local address 127.0.0.1:38062)
0811 18:54:06.069075 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.069130 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.069291 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.070231 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.070237 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.070260 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:06.070431 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.070439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.072019 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.072024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.073129 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.073139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.074099 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.074103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.074275 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.074323 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.074469 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.074605 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":58,"thread_start_us":108,"threads_started":1}
I20260811 18:54:06.074774 16987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.068477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.27.115.195:34409)
0811 18:54:06.069743 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.069755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.069773 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:06.069837 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.069841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.070010 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.070601 (+ 591us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:06.070754 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.071851 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.073274 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.073949 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.074099 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.074282 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.074505 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.074613 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1165,"thread_start_us":110,"threads_started":1}
I20260811 18:54:06.075488 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.068648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42339 (local address 127.0.0.1:51904)
0811 18:54:06.069946 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.069961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.070055 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.070590 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.070593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.070624 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:06.070746 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.070755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.071783 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.071787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.073428 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.073434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.074836 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.074840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.075077 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.075111 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.075197 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.075291 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":70,"thread_start_us":94,"threads_started":1}
I20260811 18:54:06.076087 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.069354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.27.115.194:42339)
0811 18:54:06.069704 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.069716 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.070172 (+ 456us) server_negotiation.cc:407] Connection header received
0811 18:54:06.070255 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.070270 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.070461 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.070616 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:06.070886 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.071630 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.073575 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.074501 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.074626 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.074750 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.075803 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.075904 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":158,"threads_started":1}
I20260811 18:54:06.076303 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.069277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.27.115.193:41789)
0811 18:54:06.069683 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.069690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.069714 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:54:06.069796 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.069802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.070011 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.070192 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:06.070623 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.071850 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.073575 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.074500 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.074626 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.075787 (+ 1161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.076026 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.076138 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":97,"threads_started":1}
I20260811 18:54:06.077183 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.068570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41789 (local address 127.0.0.1:54634)
0811 18:54:06.069368 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.069385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.069478 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.070339 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.070341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.070354 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:06.070476 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.070484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.071986 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.071989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.073419 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.073431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.075737 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.075743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.075858 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.075882 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.076962 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.077041 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":143,"thread_start_us":129,"threads_started":1}
I20260811 18:54:06.078539 16902 tablet_service.cc:1467] Processing CreateTablet for tablet 101e2b15598f43acb294b04f079fe8f7 (DEFAULT_TABLE table=test-table [id=2f32be9e312944fe8485e68d93859a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:06.078716 16751 tablet_service.cc:1467] Processing CreateTablet for tablet 101e2b15598f43acb294b04f079fe8f7 (DEFAULT_TABLE table=test-table [id=2f32be9e312944fe8485e68d93859a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:06.079602 16902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101e2b15598f43acb294b04f079fe8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:06.079748 16751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101e2b15598f43acb294b04f079fe8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:06.079982 16606 tablet_service.cc:1467] Processing CreateTablet for tablet 101e2b15598f43acb294b04f079fe8f7 (DEFAULT_TABLE table=test-table [id=2f32be9e312944fe8485e68d93859a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:06.081073 16606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101e2b15598f43acb294b04f079fe8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:06.087455 16992 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Bootstrap starting.
I20260811 18:54:06.088603 16993 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Bootstrap starting.
I20260811 18:54:06.089944 16992 tablet_bootstrap.cc:654] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:06.090574 16994 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Bootstrap starting.
I20260811 18:54:06.090909 16993 tablet_bootstrap.cc:654] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:06.091501 16992 log.cc:824] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:06.092123 16993 log.cc:824] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:06.092912 16994 tablet_bootstrap.cc:654] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:06.093724 16994 log.cc:824] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:06.093962 16992 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: No bootstrap required, opened a new log
I20260811 18:54:06.094029 16993 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: No bootstrap required, opened a new log
I20260811 18:54:06.094266 16992 ts_tablet_manager.cc:1397] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260811 18:54:06.094261 16993 ts_tablet_manager.cc:1397] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:54:06.095913 16994 tablet_bootstrap.cc:492] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: No bootstrap required, opened a new log
I20260811 18:54:06.096130 16994 ts_tablet_manager.cc:1397] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:54:06.097540 16993 raft_consensus.cc:348] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.097853 16993 raft_consensus.cc:374] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:06.097909 16993 raft_consensus.cc:729] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53243c233e204d93940de5fd97f49caf, State: Initialized, Role: FOLLOWER
I20260811 18:54:06.098441 16993 consensus_queue.cc:260] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.098369 16992 raft_consensus.cc:348] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.098687 16992 raft_consensus.cc:374] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:06.098750 16992 raft_consensus.cc:729] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906c3c693af242b79d2db127b90db323, State: Initialized, Role: FOLLOWER
I20260811 18:54:06.099241 16992 consensus_queue.cc:260] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.099376 16824 heartbeater.cc:503] Master 127.27.115.254:37621 was elected leader, sending a full tablet report...
I20260811 18:54:06.099668 16993 ts_tablet_manager.cc:1428] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:54:06.100243 16992 ts_tablet_manager.cc:1428] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:54:06.100247 16973 heartbeater.cc:503] Master 127.27.115.254:37621 was elected leader, sending a full tablet report...
I20260811 18:54:06.101009 16994 raft_consensus.cc:348] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.101330 16994 raft_consensus.cc:374] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:06.101383 16994 raft_consensus.cc:729] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560909cdc81b4d90832e2062f6ab0f4b, State: Initialized, Role: FOLLOWER
I20260811 18:54:06.101846 16994 consensus_queue.cc:260] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.102850 16679 heartbeater.cc:503] Master 127.27.115.254:37621 was elected leader, sending a full tablet report...
I20260811 18:54:06.103025 16994 ts_tablet_manager.cc:1428] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Time spent starting tablet: real 0.007s user 0.000s sys 0.008s
W20260811 18:54:06.247429 16680 tablet.cc:2369] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:54:06.260933 16825 tablet.cc:2369] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:54:06.261619 16974 tablet.cc:2369] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:06.327907 17000 raft_consensus.cc:482] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:54:06.328222 17000 raft_consensus.cc:504] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.329470 17000 leader_election.cc:288] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 906c3c693af242b79d2db127b90db323 (127.27.115.195:34409), 53243c233e204d93940de5fd97f49caf (127.27.115.194:42339)
I20260811 18:54:06.334252 16987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.329953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:39989 (local address 127.27.115.195:34409)
0811 18:54:06.330099 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.330106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.330549 (+ 443us) server_negotiation.cc:407] Connection header received
0811 18:54:06.330711 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.330715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.330773 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.330862 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:06.331254 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.331923 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333176 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.333612 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333736 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.333794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.333993 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.334074 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:54:06.334285 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.329952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:58801 (local address 127.27.115.194:42339)
0811 18:54:06.330098 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.330103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.330434 (+ 331us) server_negotiation.cc:407] Connection header received
0811 18:54:06.330922 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.330925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.330985 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.331079 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:06.331545 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.332167 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333237 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.333632 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333777 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.333835 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.334044 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.334159 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:54:06.334297 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.329825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42339 (local address 127.27.115.193:58801)
0811 18:54:06.330335 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.330351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.330491 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.331205 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.331209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.331233 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:06.331370 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.331378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.332287 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.332290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.333078 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.333114 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.333784 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.333893 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.333911 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.334013 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.334088 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":149,"threads_started":1}
I20260811 18:54:06.334319 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.329825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:34409 (local address 127.27.115.193:39989)
0811 18:54:06.330434 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.330446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:06.330578 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:06.330901 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:06.330904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:06.330924 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:06.331097 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.331105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.332059 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.332062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.333021 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:06.333032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.333744 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.333748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:06.333843 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.333879 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:54:06.333943 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:54:06.334033 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":218,"thread_start_us":192,"threads_started":1}
I20260811 18:54:06.335340 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "101e2b15598f43acb294b04f079fe8f7" candidate_uuid: "560909cdc81b4d90832e2062f6ab0f4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53243c233e204d93940de5fd97f49caf" is_pre_election: true
I20260811 18:54:06.335446 16925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "101e2b15598f43acb294b04f079fe8f7" candidate_uuid: "560909cdc81b4d90832e2062f6ab0f4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906c3c693af242b79d2db127b90db323" is_pre_election: true
I20260811 18:54:06.335732 16776 raft_consensus.cc:2454] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560909cdc81b4d90832e2062f6ab0f4b in term 0.
I20260811 18:54:06.335777 16925 raft_consensus.cc:2454] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560909cdc81b4d90832e2062f6ab0f4b in term 0.
I20260811 18:54:06.336292 16555 leader_election.cc:302] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53243c233e204d93940de5fd97f49caf, 560909cdc81b4d90832e2062f6ab0f4b, 906c3c693af242b79d2db127b90db323; no voters:
I20260811 18:54:06.336619 17000 raft_consensus.cc:2781] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:06.336706 17000 raft_consensus.cc:482] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:06.336738 17000 raft_consensus.cc:3037] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:06.337862 17000 raft_consensus.cc:504] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.338338 17000 leader_election.cc:288] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [CANDIDATE]: Term 1 election: Requested vote from peers 906c3c693af242b79d2db127b90db323 (127.27.115.195:34409), 53243c233e204d93940de5fd97f49caf (127.27.115.194:42339)
I20260811 18:54:06.338791 16925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "101e2b15598f43acb294b04f079fe8f7" candidate_uuid: "560909cdc81b4d90832e2062f6ab0f4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906c3c693af242b79d2db127b90db323"
I20260811 18:54:06.338799 16776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "101e2b15598f43acb294b04f079fe8f7" candidate_uuid: "560909cdc81b4d90832e2062f6ab0f4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53243c233e204d93940de5fd97f49caf"
I20260811 18:54:06.338943 16925 raft_consensus.cc:3037] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:06.338953 16776 raft_consensus.cc:3037] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:06.340081 16925 raft_consensus.cc:2454] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560909cdc81b4d90832e2062f6ab0f4b in term 1.
I20260811 18:54:06.340094 16776 raft_consensus.cc:2454] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560909cdc81b4d90832e2062f6ab0f4b in term 1.
I20260811 18:54:06.340462 16553 leader_election.cc:302] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560909cdc81b4d90832e2062f6ab0f4b, 906c3c693af242b79d2db127b90db323; no voters:
I20260811 18:54:06.340751 17000 raft_consensus.cc:2781] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:06.341022 17000 raft_consensus.cc:686] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 1 LEADER]: Becoming Leader. State: Replica: 560909cdc81b4d90832e2062f6ab0f4b, State: Running, Role: LEADER
I20260811 18:54:06.341389 17000 consensus_queue.cc:237] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } }
I20260811 18:54:06.345217 16466 catalog_manager.cc:5495] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 560909cdc81b4d90832e2062f6ab0f4b (127.27.115.193). New cstate: current_term: 1 leader_uuid: "560909cdc81b4d90832e2062f6ab0f4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560909cdc81b4d90832e2062f6ab0f4b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 41789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 } health_report { overall_health: UNKNOWN } } }
I20260811 18:54:06.378647 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.373962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.27.115.193:41789)
0811 18:54:06.374219 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.374223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.374237 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:54:06.374316 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.374318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.374373 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.374475 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:06.374771 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.375298 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.376589 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.376821 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.376921 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.377499 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:06.377984 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:06.378192 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.378453 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.378501 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 18:54:06.385283 16776 raft_consensus.cc:1261] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 1 FOLLOWER]: Refusing update from remote peer 560909cdc81b4d90832e2062f6ab0f4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:06.385289 16925 raft_consensus.cc:1261] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 1 FOLLOWER]: Refusing update from remote peer 560909cdc81b4d90832e2062f6ab0f4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:06.386425 17004 consensus_queue.cc:1035] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "906c3c693af242b79d2db127b90db323" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:06.386672 17000 consensus_queue.cc:1035] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "53243c233e204d93940de5fd97f49caf" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:06.408425 16466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:06.408624 16466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:06.412261 16466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cbece0b3a2743beb82d1f40af0e57b0: Sending DeleteTablet for 3 replicas of tablet 101e2b15598f43acb294b04f079fe8f7
I20260811 18:54:06.413455 16606 tablet_service.cc:1514] Processing DeleteTablet for tablet 101e2b15598f43acb294b04f079fe8f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:06 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:54634
I20260811 18:54:06.413532 16902 tablet_service.cc:1514] Processing DeleteTablet for tablet 101e2b15598f43acb294b04f079fe8f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:06 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:38062
I20260811 18:54:06.413537 16751 tablet_service.cc:1514] Processing DeleteTablet for tablet 101e2b15598f43acb294b04f079fe8f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:06 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:51904
I20260811 18:54:06.414548 17017 tablet_replica.cc:330] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b: stopping tablet replica
I20260811 18:54:06.414973 17017 raft_consensus.cc:2229] T 101e2b15598f43acb294b04f079fe8f7 P 560909cdc81b4d90832e2062f6ab0f4b [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:54:06.415683 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16535
I20260811 18:54:06.421625 17019 tablet_replica.cc:330] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: stopping tablet replica
I20260811 18:54:06.422087 17019 raft_consensus.cc:2229] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:06.422593 17019 raft_consensus.cc:2258] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:06.423180 17018 tablet_replica.cc:330] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: stopping tablet replica
I20260811 18:54:06.423623 17018 raft_consensus.cc:2229] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:06.423966 17018 raft_consensus.cc:2258] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:06.424690 17019 ts_tablet_manager.cc:1905] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:06.425406 17018 ts_tablet_manager.cc:1905] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:06.428121 17018 ts_tablet_manager.cc:1918] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:06.428284 17018 log.cc:1196] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/101e2b15598f43acb294b04f079fe8f7
I20260811 18:54:06.428627 17018 ts_tablet_manager.cc:1939] T 101e2b15598f43acb294b04f079fe8f7 P 906c3c693af242b79d2db127b90db323: Deleting consensus metadata
I20260811 18:54:06.428817 17019 ts_tablet_manager.cc:1918] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:06.428951 17019 log.cc:1196] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/101e2b15598f43acb294b04f079fe8f7
I20260811 18:54:06.429332 17019 ts_tablet_manager.cc:1939] T 101e2b15598f43acb294b04f079fe8f7 P 53243c233e204d93940de5fd97f49caf: Deleting consensus metadata
I20260811 18:54:06.430254 16452 catalog_manager.cc:4841] TS 906c3c693af242b79d2db127b90db323 (127.27.115.195:34409): tablet 101e2b15598f43acb294b04f079fe8f7 (table test-table [id=2f32be9e312944fe8485e68d93859a72]) successfully deleted
I20260811 18:54:06.430753 16454 catalog_manager.cc:4841] TS 53243c233e204d93940de5fd97f49caf (127.27.115.194:42339): tablet 101e2b15598f43acb294b04f079fe8f7 (table test-table [id=2f32be9e312944fe8485e68d93859a72]) successfully deleted
W20260811 18:54:06.431420 16451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:41789 (error 108)
I20260811 18:54:06.431579 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16685
I20260811 18:54:06.436110 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.435758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:41789 (local address 127.0.0.1:50408)
0811 18:54:06.435918 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:06.436015 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:41789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 18:54:06.436420 16451 catalog_manager.cc:4584] TS 560909cdc81b4d90832e2062f6ab0f4b (127.27.115.193:41789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 101e2b15598f43acb294b04f079fe8f7: Network error: Client connection negotiation failed: client connection to 127.27.115.193:41789: connect: Connection refused (error 111)
I20260811 18:54:06.441205 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16829
I20260811 18:54:06.454487 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 16433
2026-08-11T18:54:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2761 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T18:54:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:06Z Disabled control of system clock
I20260811 18:54:06.494310 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36855
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:36855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:06.622645 17024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:06.622956 17024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:06.623016 17024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:06.627847 17024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:06.627944 17024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:06.627960 17024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:06.627978 17024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:06.627995 17024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:06.633724 17024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:36855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:06.635128 17024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:06.636859 17024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:06.647367 17033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:06.647359 17031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:06.647372 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:06.647648 17024 server_base.cc:1034] running on GCE node
I20260811 18:54:06.648342 17024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:06.649618 17024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:06.650825 17024 hybrid_clock.cc:648] HybridClock initialized: now 1786474446650806 us; error 39 us; skew 500 ppm
I20260811 18:54:06.653366 17024 webserver.cc:460] Webserver started at http://127.27.115.254:43843/ using document root <none> and password file <none>
I20260811 18:54:06.654229 17024 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:06.654333 17024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:06.654620 17024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:06.658082 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "599e082c4a814258a996c85cb5fd856b"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "2411e3b28553822d00f55a59da78fd7d"
server_key_iv: "0391018f632bc9b7acfc436892771302"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.659188 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "599e082c4a814258a996c85cb5fd856b"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "2411e3b28553822d00f55a59da78fd7d"
server_key_iv: "0391018f632bc9b7acfc436892771302"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.667786 17024 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 18:54:06.671769 17039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:06.673503 17024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:54:06.673686 17024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "599e082c4a814258a996c85cb5fd856b"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "2411e3b28553822d00f55a59da78fd7d"
server_key_iv: "0391018f632bc9b7acfc436892771302"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.673846 17024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:06.705901 17024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:06.706823 17024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:06.707057 17024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:06.716181 17024 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:36855
I20260811 18:54:06.716305 17099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:36855 every 8 connection(s)
I20260811 18:54:06.717553 17024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:06.721748 17100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:06.724519 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17024
I20260811 18:54:06.724659 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:06.725281 28111 external_mini_cluster.cc:1428] Setting key 0e3bc998af79a8072adf7073f052d757
I20260811 18:54:06.729723 17100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: Bootstrap starting.
I20260811 18:54:06.733048 17100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:06.734015 17100 log.cc:824] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:06.736825 17100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: No bootstrap required, opened a new log
I20260811 18:54:06.737154 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:06.727219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54856 (local address 127.27.115.254:36855)
0811 18:54:06.728332 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:06.728353 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:06.728397 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:54:06.729082 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:06.729101 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:06.729584 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:06.730003 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:06.730576 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.731633 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.732844 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:06.733440 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:06.733632 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:06.734785 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:06.734791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:06.734808 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:06.735354 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:06.735435 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:06.735466 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:06.735578 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:06.735804 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:54:06.736100 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":196,"threads_started":1}
I20260811 18:54:06.741257 17100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } }
I20260811 18:54:06.741562 17100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:06.741597 17100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599e082c4a814258a996c85cb5fd856b, State: Initialized, Role: FOLLOWER
I20260811 18:54:06.742012 17100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } }
I20260811 18:54:06.742305 17100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:06.742422 17100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:06.742520 17100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:06.743732 17100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } }
I20260811 18:54:06.744130 17100 leader_election.cc:302] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 599e082c4a814258a996c85cb5fd856b; no voters:
I20260811 18:54:06.744536 17100 leader_election.cc:288] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:06.744760 17106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:06.745308 17106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [term 1 LEADER]: Becoming Leader. State: Replica: 599e082c4a814258a996c85cb5fd856b, State: Running, Role: LEADER
I20260811 18:54:06.745780 17106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } }
I20260811 18:54:06.746526 17100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:06.747969 17107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "599e082c4a814258a996c85cb5fd856b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } } }
I20260811 18:54:06.747977 17108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 599e082c4a814258a996c85cb5fd856b. Latest consensus state: current_term: 1 leader_uuid: "599e082c4a814258a996c85cb5fd856b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599e082c4a814258a996c85cb5fd856b" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36855 } } }
I20260811 18:54:06.748163 17108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:06.748162 17107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:06.748878 17115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:06.752300 17115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:06.759729 17115 catalog_manager.cc:1347] Generated new cluster ID: d0ab4c085c444ee98a31d58fb644c544
I20260811 18:54:06.759842 17115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:06.771250 17115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:06.772605 17115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:06.780472 17115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: Generated new TSK 0
I20260811 18:54:06.781464 17115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:06.791787 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--builtin_ntp_servers=127.27.115.212:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:06.926668 17125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:06.926985 17125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:06.927034 17125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:06.931439 17125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:06.931533 17125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:06.931593 17125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:06.936908 17125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:06.938241 17125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:06.939921 17125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:06.952109 17131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:06.952116 17130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:06.952112 17133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:06.952560 17125 server_base.cc:1034] running on GCE node
I20260811 18:54:06.953153 17125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:06.954036 17125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:06.955277 17125 hybrid_clock.cc:648] HybridClock initialized: now 1786474446955261 us; error 39 us; skew 500 ppm
I20260811 18:54:06.957592 17125 webserver.cc:460] Webserver started at http://127.27.115.193:46707/ using document root <none> and password file <none>
I20260811 18:54:06.958410 17125 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:06.958523 17125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:06.958781 17125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:06.961081 17125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "f0617ddbaaea4a80a8869173edf9693c"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "17ac6d38b5942ce5811d5936ee8fdcda"
server_key_iv: "ab66d5b5b3d23aa3138919a89da07083"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.961719 17125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "f0617ddbaaea4a80a8869173edf9693c"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "17ac6d38b5942ce5811d5936ee8fdcda"
server_key_iv: "ab66d5b5b3d23aa3138919a89da07083"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.965804 17125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:54:06.968590 17139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:06.970082 17125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:06.970279 17125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "f0617ddbaaea4a80a8869173edf9693c"
format_stamp: "Formatted at 2026-08-11 18:54:06 on dist-test-slave-9jjr"
server_key: "17ac6d38b5942ce5811d5936ee8fdcda"
server_key_iv: "ab66d5b5b3d23aa3138919a89da07083"
server_key_version: "encryptionkey@0"
I20260811 18:54:06.970417 17125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:07.003401 17125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:07.004289 17125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:07.004552 17125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:07.005280 17125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:07.006477 17125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:07.006561 17125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.006644 17125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:07.006688 17125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.017727 17125 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:40287
I20260811 18:54:07.017745 17263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:40287 every 8 connection(s)
I20260811 18:54:07.018818 17125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:07.021363 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17125
I20260811 18:54:07.021497 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:07.021839 28111 external_mini_cluster.cc:1428] Setting key 3d8647129fbe06cfab37731cc4a5f6f0
I20260811 18:54:07.024713 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--builtin_ntp_servers=127.27.115.212:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:07.030675 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.020994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:37057 (local address 127.27.115.254:36855)
0811 18:54:07.021193 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.021198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.022551 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 18:54:07.024353 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.024357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.024415 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.024526 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:07.025637 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.026420 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.027996 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.028260 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.028396 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.029928 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:07.029932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:07.029935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:07.029957 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:07.029985 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:07.029995 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:07.030092 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.030372 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.030466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:54:07.032049 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.021461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36855 (local address 127.27.115.193:37057)
0811 18:54:07.022370 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.022431 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.024095 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.024730 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.024738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.024834 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:07.025464 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.025478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.026652 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.026668 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.027762 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.027779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.028470 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.028475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.028685 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.029508 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:07.029510 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:07.029565 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:07.029656 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:07.030167 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:07.030201 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:07.030208 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.030369 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.030763 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":168,"threads_started":1}
I20260811 18:54:07.033947 17264 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36855
I20260811 18:54:07.034386 17264 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:07.035255 17264 heartbeater.cc:511] Master 127.27.115.254:36855 requested a full tablet report, sending...
I20260811 18:54:07.037640 17060 ts_manager.cc:194] Registered new tserver with Master: f0617ddbaaea4a80a8869173edf9693c (127.27.115.193:40287)
I20260811 18:54:07.039611 17060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.193:37057
W20260811 18:54:07.151428 17269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:07.151796 17269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:07.151844 17269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:07.156476 17269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:07.156564 17269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:07.156632 17269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:07.161772 17269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:07.163179 17269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:07.164760 17269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:07.175987 17275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:07.175987 17277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:07.176036 17274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:07.176405 17269 server_base.cc:1034] running on GCE node
I20260811 18:54:07.176867 17269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:07.177646 17269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:07.178849 17269 hybrid_clock.cc:648] HybridClock initialized: now 1786474447178828 us; error 33 us; skew 500 ppm
I20260811 18:54:07.180984 17269 webserver.cc:460] Webserver started at http://127.27.115.194:45819/ using document root <none> and password file <none>
I20260811 18:54:07.181725 17269 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:07.181799 17269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:07.182029 17269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:07.184346 17269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "24fa25cb28424178977c99c770559fb2"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "11e51209efe5a9f1bbebcb41f89de5fa"
server_key_iv: "8d98ca2eb4dc954067f517faa187fdd0"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.184968 17269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "24fa25cb28424178977c99c770559fb2"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "11e51209efe5a9f1bbebcb41f89de5fa"
server_key_iv: "8d98ca2eb4dc954067f517faa187fdd0"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.188936 17269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:54:07.191668 17283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.193032 17269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:54:07.193233 17269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "24fa25cb28424178977c99c770559fb2"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "11e51209efe5a9f1bbebcb41f89de5fa"
server_key_iv: "8d98ca2eb4dc954067f517faa187fdd0"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.193356 17269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:07.219151 17269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:07.219944 17269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:07.220176 17269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:07.220845 17269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:07.221870 17269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:07.221920 17269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.221963 17269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:07.221979 17269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.231734 17269 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:36639
I20260811 18:54:07.231773 17406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:36639 every 8 connection(s)
I20260811 18:54:07.232733 17269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:07.235049 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17269
I20260811 18:54:07.235188 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:07.235543 28111 external_mini_cluster.cc:1428] Setting key 3bcf3823c5cf83db91c1e16bd2b7cfd0
I20260811 18:54:07.237834 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--builtin_ntp_servers=127.27.115.212:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:07.243954 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.234837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:52709 (local address 127.27.115.254:36855)
0811 18:54:07.234981 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.234985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.236523 (+ 1538us) server_negotiation.cc:407] Connection header received
0811 18:54:07.238395 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.238399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.238463 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.238546 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:07.239601 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.240167 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.241470 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.241719 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.241849 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.243386 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:07.243389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:07.243392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:07.243408 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:07.243430 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:07.243439 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:07.243525 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.243707 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.243787 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:54:07.245003 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.235413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36855 (local address 127.27.115.194:52709)
0811 18:54:07.236347 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.236409 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.238079 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.238747 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.238755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.238852 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:07.239435 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.239450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.240337 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.240340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.241319 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.241331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.241921 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.241926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.242191 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.242987 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:07.242989 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:07.243040 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:07.243183 (+ 143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:07.243528 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:07.243542 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:07.243549 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.243707 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.244087 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":174,"threads_started":1}
I20260811 18:54:07.246443 17407 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36855
I20260811 18:54:07.246747 17407 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:07.247426 17407 heartbeater.cc:511] Master 127.27.115.254:36855 requested a full tablet report, sending...
I20260811 18:54:07.248847 17060 ts_manager.cc:194] Registered new tserver with Master: 24fa25cb28424178977c99c770559fb2 (127.27.115.194:36639)
I20260811 18:54:07.249795 17060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.194:52709
W20260811 18:54:07.364809 17410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:07.365207 17410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:07.365252 17410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:07.369323 17410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:07.369390 17410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:07.369446 17410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:07.374399 17410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:07.375705 17410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:07.377177 17410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:07.387694 17415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:07.387662 17417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:07.387714 17419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:07.387902 17410 server_base.cc:1034] running on GCE node
I20260811 18:54:07.388444 17410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:07.389227 17410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:07.390424 17410 hybrid_clock.cc:648] HybridClock initialized: now 1786474447390397 us; error 33 us; skew 500 ppm
I20260811 18:54:07.392762 17410 webserver.cc:460] Webserver started at http://127.27.115.195:39897/ using document root <none> and password file <none>
I20260811 18:54:07.393388 17410 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:07.393455 17410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:07.393630 17410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:07.396024 17410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "4fcad3d34bb14342bb347796e63d3156"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "ac51dd6cb67ecf063302d84fcf793388"
server_key_iv: "aa39957af26b19574189fc42933e0d74"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.396659 17410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "4fcad3d34bb14342bb347796e63d3156"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "ac51dd6cb67ecf063302d84fcf793388"
server_key_iv: "aa39957af26b19574189fc42933e0d74"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.401002 17410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:54:07.403864 17425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.405427 17410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:54:07.405570 17410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "4fcad3d34bb14342bb347796e63d3156"
format_stamp: "Formatted at 2026-08-11 18:54:07 on dist-test-slave-9jjr"
server_key: "ac51dd6cb67ecf063302d84fcf793388"
server_key_iv: "aa39957af26b19574189fc42933e0d74"
server_key_version: "encryptionkey@0"
I20260811 18:54:07.405745 17410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:07.428033 17410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:07.428828 17410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:07.429005 17410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:07.429661 17410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:07.430780 17410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:07.430832 17410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.430876 17410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:07.430893 17410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:07.441908 17410 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:41037
I20260811 18:54:07.441936 17551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:41037 every 8 connection(s)
I20260811 18:54:07.443024 17410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:07.446795 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17410
I20260811 18:54:07.446972 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:07.447379 28111 external_mini_cluster.cc:1428] Setting key 867bf7469c54e52c1928f265e55319a2
I20260811 18:54:07.454931 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.445217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:42749 (local address 127.27.115.254:36855)
0811 18:54:07.445408 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.445414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.446836 (+ 1422us) server_negotiation.cc:407] Connection header received
0811 18:54:07.448903 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.448907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.448969 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.449063 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:07.450324 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.450879 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.452252 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.452498 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.452694 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.454111 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:07.454115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:07.454117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:07.454216 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:07.454250 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:07.454260 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:07.454328 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.454696 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.454765 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:54:07.455856 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.445759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36855 (local address 127.27.115.195:42749)
0811 18:54:07.446648 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.446695 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.448615 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.449294 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.449305 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.449426 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:07.450121 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.450158 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.451056 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.451059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.452060 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.452073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.452728 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.452732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.452900 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.453782 (+ 882us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:07.453784 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:07.453831 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:07.453910 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:07.454446 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:07.454459 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:07.454466 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.454612 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.454882 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":126,"threads_started":1}
I20260811 18:54:07.457192 17552 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36855
I20260811 18:54:07.457562 17552 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:07.458207 17552 heartbeater.cc:511] Master 127.27.115.254:36855 requested a full tablet report, sending...
I20260811 18:54:07.459367 17060 ts_manager.cc:194] Registered new tserver with Master: 4fcad3d34bb14342bb347796e63d3156 (127.27.115.195:41037)
I20260811 18:54:07.460005 17060 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.195:42749
I20260811 18:54:07.461799 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:07.468420 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.463953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.27.115.254:36855)
0811 18:54:07.464110 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.464117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.464293 (+ 176us) server_negotiation.cc:407] Connection header received
0811 18:54:07.464379 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.464383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.464445 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.464555 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:07.465044 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.465565 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.466533 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.466740 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.466829 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.467748 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:07.467751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:07.467753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:07.467766 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:07.467804 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:07.467812 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:07.467909 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.468193 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.468241 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:54:07.471776 17060 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:07.475275 17060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:07.491743 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.485244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:41037 (local address 127.0.0.1:58804)
0811 18:54:07.486195 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.486251 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.486413 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.487156 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.487161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.487186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:07.487332 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.487339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.488824 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.488828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.490103 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.490114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.491076 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.491080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.491220 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.491269 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.491412 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.491546 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":77,"thread_start_us":122,"threads_started":1}
I20260811 18:54:07.491791 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.485617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58804 (local address 127.27.115.195:41037)
0811 18:54:07.486000 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.486014 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.486580 (+ 566us) server_negotiation.cc:407] Connection header received
0811 18:54:07.486694 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.486703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.486928 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.487118 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:07.487456 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.488681 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.490270 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.490932 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.491091 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.491291 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.491451 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.491571 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":145,"threads_started":1}
I20260811 18:54:07.492146 17566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.485279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:36639 (local address 127.0.0.1:35444)
0811 18:54:07.486419 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.486434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.486536 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.487572 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.487574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.487594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:07.487672 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.487677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.488778 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.488784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.490617 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.490630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.491735 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.491739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.491843 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.491871 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.491933 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.492005 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":169,"thread_start_us":99,"threads_started":1}
I20260811 18:54:07.493077 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.486037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40287 (local address 127.0.0.1:46102)
0811 18:54:07.486949 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.486965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.487071 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.487745 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.487750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.487772 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:07.487865 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.487872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.489284 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.489288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.490796 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.490813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.492296 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.492302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.492714 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.492738 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.492853 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.492932 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":89,"thread_start_us":150,"threads_started":1}
I20260811 18:54:07.493083 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.486731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.27.115.194:36639)
0811 18:54:07.487183 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.487195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.487217 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:54:07.487282 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.487287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.487429 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.487584 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:07.487799 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.488619 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.490789 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.491579 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.492574 (+ 995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.492751 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.492810 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.492917 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":129,"threads_started":1}
I20260811 18:54:07.493217 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.486540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.27.115.193:40287)
0811 18:54:07.486879 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.486885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.487066 (+ 181us) server_negotiation.cc:407] Connection header received
0811 18:54:07.487236 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.487242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.487430 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.487583 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:07.488000 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.489151 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.490939 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.491870 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.492009 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.492218 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.492897 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.493015 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":110,"threads_started":1}
I20260811 18:54:07.496229 17334 tablet_service.cc:1467] Processing CreateTablet for tablet dce567959b784daeb7763c9d68faec20 (DEFAULT_TABLE table=test-table [id=9ef77ada98c047e69d4c7b7ee48c2b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:07.496110 17190 tablet_service.cc:1467] Processing CreateTablet for tablet dce567959b784daeb7763c9d68faec20 (DEFAULT_TABLE table=test-table [id=9ef77ada98c047e69d4c7b7ee48c2b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:07.496109 17480 tablet_service.cc:1467] Processing CreateTablet for tablet dce567959b784daeb7763c9d68faec20 (DEFAULT_TABLE table=test-table [id=9ef77ada98c047e69d4c7b7ee48c2b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:07.497560 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce567959b784daeb7763c9d68faec20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:07.497562 17190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce567959b784daeb7763c9d68faec20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:07.497562 17480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce567959b784daeb7763c9d68faec20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:07.506078 17569 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Bootstrap starting.
I20260811 18:54:07.507156 17570 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Bootstrap starting.
I20260811 18:54:07.508850 17569 tablet_bootstrap.cc:654] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:07.509204 17570 tablet_bootstrap.cc:654] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:07.509783 17569 log.cc:824] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:07.510074 17570 log.cc:824] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:07.511104 17571 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Bootstrap starting.
I20260811 18:54:07.512679 17569 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: No bootstrap required, opened a new log
I20260811 18:54:07.512833 17570 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: No bootstrap required, opened a new log
I20260811 18:54:07.512954 17569 ts_tablet_manager.cc:1397] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 18:54:07.513051 17570 ts_tablet_manager.cc:1397] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:54:07.513777 17571 tablet_bootstrap.cc:654] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:07.514792 17571 log.cc:824] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:07.516922 17569 raft_consensus.cc:348] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.517207 17569 raft_consensus.cc:374] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:07.517274 17569 raft_consensus.cc:729] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24fa25cb28424178977c99c770559fb2, State: Initialized, Role: FOLLOWER
I20260811 18:54:07.517414 17571 tablet_bootstrap.cc:492] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: No bootstrap required, opened a new log
I20260811 18:54:07.517431 17570 raft_consensus.cc:348] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.517669 17571 ts_tablet_manager.cc:1397] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:54:07.517694 17570 raft_consensus.cc:374] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:07.517742 17570 raft_consensus.cc:729] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0617ddbaaea4a80a8869173edf9693c, State: Initialized, Role: FOLLOWER
I20260811 18:54:07.517863 17569 consensus_queue.cc:260] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.518326 17570 consensus_queue.cc:260] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.519388 17569 ts_tablet_manager.cc:1428] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 18:54:07.519505 17570 ts_tablet_manager.cc:1428] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:54:07.519979 17407 heartbeater.cc:503] Master 127.27.115.254:36855 was elected leader, sending a full tablet report...
I20260811 18:54:07.519979 17264 heartbeater.cc:503] Master 127.27.115.254:36855 was elected leader, sending a full tablet report...
W20260811 18:54:07.520210 17265 tablet.cc:2369] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:07.524026 17571 raft_consensus.cc:348] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.524349 17571 raft_consensus.cc:374] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:07.524422 17571 raft_consensus.cc:729] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad3d34bb14342bb347796e63d3156, State: Initialized, Role: FOLLOWER
I20260811 18:54:07.524959 17571 consensus_queue.cc:260] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.525977 17552 heartbeater.cc:503] Master 127.27.115.254:36855 was elected leader, sending a full tablet report...
I20260811 18:54:07.527355 17571 ts_tablet_manager.cc:1428] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 18:54:07.579340 17579 raft_consensus.cc:482] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:54:07.579609 17579 raft_consensus.cc:504] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.580899 17579 leader_election.cc:288] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24fa25cb28424178977c99c770559fb2 (127.27.115.194:36639), f0617ddbaaea4a80a8869173edf9693c (127.27.115.193:40287)
I20260811 18:54:07.585074 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.581241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:36639 (local address 127.27.115.195:37393)
0811 18:54:07.581400 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.581419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.581531 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.581847 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.581851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.581873 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:07.582012 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.582020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.582816 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.582821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.583928 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.583939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.584708 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.584711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.584811 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.584832 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.584894 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.584956 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 18:54:07.585166 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.581355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:37393 (local address 127.27.115.194:36639)
0811 18:54:07.581536 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.581543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.581563 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:54:07.581632 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.581638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.581699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.581832 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:07.582169 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.582692 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.584077 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.584588 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.584713 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.584766 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.584933 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.585010 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:54:07.585695 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.581747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:48121 (local address 127.27.115.193:40287)
0811 18:54:07.581980 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.581989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.582027 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:54:07.582309 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.582312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.582361 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.582450 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:07.582738 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.583267 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.584482 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.585052 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.585174 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.585228 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.585508 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.585585 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260811 18:54:07.585691 17580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.581566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40287 (local address 127.27.115.195:48121)
0811 18:54:07.581948 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.581963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:07.582099 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:07.582465 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:07.582469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:07.582486 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:07.582620 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.582627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.583403 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.583407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.584358 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:07.584365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.585206 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.585210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:07.585366 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.585396 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:54:07.585486 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:54:07.585553 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":171,"threads_started":1}
I20260811 18:54:07.586015 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce567959b784daeb7763c9d68faec20" candidate_uuid: "4fcad3d34bb14342bb347796e63d3156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fa25cb28424178977c99c770559fb2" is_pre_election: true
I20260811 18:54:07.586432 17357 raft_consensus.cc:2454] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad3d34bb14342bb347796e63d3156 in term 0.
I20260811 18:54:07.586411 17213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce567959b784daeb7763c9d68faec20" candidate_uuid: "4fcad3d34bb14342bb347796e63d3156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0617ddbaaea4a80a8869173edf9693c" is_pre_election: true
I20260811 18:54:07.586691 17213 raft_consensus.cc:2454] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad3d34bb14342bb347796e63d3156 in term 0.
I20260811 18:54:07.587028 17428 leader_election.cc:302] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24fa25cb28424178977c99c770559fb2, 4fcad3d34bb14342bb347796e63d3156; no voters:
I20260811 18:54:07.587384 17579 raft_consensus.cc:2781] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:07.587471 17579 raft_consensus.cc:482] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:07.587500 17579 raft_consensus.cc:3037] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:07.588763 17579 raft_consensus.cc:504] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.589177 17579 leader_election.cc:288] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [CANDIDATE]: Term 1 election: Requested vote from peers 24fa25cb28424178977c99c770559fb2 (127.27.115.194:36639), f0617ddbaaea4a80a8869173edf9693c (127.27.115.193:40287)
I20260811 18:54:07.589715 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce567959b784daeb7763c9d68faec20" candidate_uuid: "4fcad3d34bb14342bb347796e63d3156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24fa25cb28424178977c99c770559fb2"
I20260811 18:54:07.589772 17213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce567959b784daeb7763c9d68faec20" candidate_uuid: "4fcad3d34bb14342bb347796e63d3156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0617ddbaaea4a80a8869173edf9693c"
I20260811 18:54:07.589879 17357 raft_consensus.cc:3037] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:07.589916 17213 raft_consensus.cc:3037] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:07.591120 17213 raft_consensus.cc:2454] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad3d34bb14342bb347796e63d3156 in term 1.
I20260811 18:54:07.591122 17357 raft_consensus.cc:2454] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad3d34bb14342bb347796e63d3156 in term 1.
I20260811 18:54:07.591595 17428 leader_election.cc:302] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24fa25cb28424178977c99c770559fb2, 4fcad3d34bb14342bb347796e63d3156; no voters:
I20260811 18:54:07.591805 17579 raft_consensus.cc:2781] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:07.592072 17579 raft_consensus.cc:686] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 1 LEADER]: Becoming Leader. State: Replica: 4fcad3d34bb14342bb347796e63d3156, State: Running, Role: LEADER
I20260811 18:54:07.592458 17579 consensus_queue.cc:237] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } }
I20260811 18:54:07.596074 17060 catalog_manager.cc:5495] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fcad3d34bb14342bb347796e63d3156 (127.27.115.195). New cstate: current_term: 1 leader_uuid: "4fcad3d34bb14342bb347796e63d3156" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fcad3d34bb14342bb347796e63d3156" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260811 18:54:07.621948 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.618051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.27.115.195:41037)
0811 18:54:07.618314 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:07.618318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:07.618334 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:54:07.618413 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:07.618415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:07.618473 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:07.618581 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:07.618873 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.619387 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.620325 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:07.620584 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:07.620685 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:07.620968 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:07.621410 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:07.621534 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:07.621701 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 18:54:07.621781 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 18:54:07.628536 17213 raft_consensus.cc:1261] T dce567959b784daeb7763c9d68faec20 P f0617ddbaaea4a80a8869173edf9693c [term 1 FOLLOWER]: Refusing update from remote peer 4fcad3d34bb14342bb347796e63d3156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:07.628536 17357 raft_consensus.cc:1261] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 1 FOLLOWER]: Refusing update from remote peer 4fcad3d34bb14342bb347796e63d3156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:07.629329 17579 consensus_queue.cc:1035] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0617ddbaaea4a80a8869173edf9693c" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:07.629530 17581 consensus_queue.cc:1035] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24fa25cb28424178977c99c770559fb2" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 36639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:07.640805 17585 mvcc.cc:205] Tried to move back new op lower bound from 7317399337478467584 to 7317399337342177280. Current Snapshot: MvccSnapshot[applied={T|T < 7317399337478467584 or (T in {7317399337478467584})}]
I20260811 18:54:07.649003 17058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:07.649256 17058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:07.652195 17058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 599e082c4a814258a996c85cb5fd856b: Sending DeleteTablet for 3 replicas of tablet dce567959b784daeb7763c9d68faec20
I20260811 18:54:07.653191 17480 tablet_service.cc:1514] Processing DeleteTablet for tablet dce567959b784daeb7763c9d68faec20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:07 UTC) from {username='slave'} at 127.0.0.1:58804
I20260811 18:54:07.653316 17190 tablet_service.cc:1514] Processing DeleteTablet for tablet dce567959b784daeb7763c9d68faec20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:07 UTC) from {username='slave'} at 127.0.0.1:46102
I20260811 18:54:07.654079 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17125
I20260811 18:54:07.654106 17334 tablet_service.cc:1514] Processing DeleteTablet for tablet dce567959b784daeb7763c9d68faec20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:07 UTC) from {username='slave'} at 127.0.0.1:35444
I20260811 18:54:07.654150 17595 tablet_replica.cc:330] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: stopping tablet replica
I20260811 18:54:07.657476 17595 raft_consensus.cc:2229] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:54:07.658036 17595 raft_consensus.cc:2258] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:07.659660 17598 tablet_replica.cc:330] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: stopping tablet replica
I20260811 18:54:07.660058 17598 raft_consensus.cc:2229] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:07.660454 17598 raft_consensus.cc:2258] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:07.660681 17595 ts_tablet_manager.cc:1905] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:07.662395 17598 ts_tablet_manager.cc:1905] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:07.665081 17595 ts_tablet_manager.cc:1918] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:07.665323 17595 log.cc:1196] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/dce567959b784daeb7763c9d68faec20
I20260811 18:54:07.665345 17598 ts_tablet_manager.cc:1918] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:07.665458 17598 log.cc:1196] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/dce567959b784daeb7763c9d68faec20
I20260811 18:54:07.665815 17598 ts_tablet_manager.cc:1939] T dce567959b784daeb7763c9d68faec20 P 24fa25cb28424178977c99c770559fb2: Deleting consensus metadata
I20260811 18:54:07.665908 17595 ts_tablet_manager.cc:1939] T dce567959b784daeb7763c9d68faec20 P 4fcad3d34bb14342bb347796e63d3156: Deleting consensus metadata
W20260811 18:54:07.667312 17043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:40287 (error 108)
I20260811 18:54:07.667367 17044 catalog_manager.cc:4841] TS 4fcad3d34bb14342bb347796e63d3156 (127.27.115.195:41037): tablet dce567959b784daeb7763c9d68faec20 (table test-table [id=9ef77ada98c047e69d4c7b7ee48c2b4d]) successfully deleted
I20260811 18:54:07.667886 17043 catalog_manager.cc:4841] TS 24fa25cb28424178977c99c770559fb2 (127.27.115.194:36639): tablet dce567959b784daeb7763c9d68faec20 (table test-table [id=9ef77ada98c047e69d4c7b7ee48c2b4d]) successfully deleted
I20260811 18:54:07.668183 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17269
I20260811 18:54:07.675869 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:07.675359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:40287 (local address 127.0.0.1:46118)
0811 18:54:07.675610 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:07.675730 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260811 18:54:07.676137 17043 catalog_manager.cc:4584] TS f0617ddbaaea4a80a8869173edf9693c (127.27.115.193:40287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dce567959b784daeb7763c9d68faec20: Network error: Client connection negotiation failed: client connection to 127.27.115.193:40287: connect: Connection refused (error 111)
I20260811 18:54:07.681429 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17410
I20260811 18:54:07.692133 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17024
2026-08-11T18:54:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1235 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:07 dist-test-slave-9jjr krb5kdc[17602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:07 dist-test-slave-9jjr krb5kdc[17602](info): set up 2 sockets
Aug 11 18:54:07 dist-test-slave-9jjr krb5kdc[17602](info): commencing operation
krb5kdc: starting...
W20260811 18:54:08.819733 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:08 dist-test-slave-9jjr krb5kdc[17602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:08.965988 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44821
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:09.088402 17615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:09.088826 17615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:09.088943 17615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:09.093644 17615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:09.093794 17615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:09.093837 17615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:09.093880 17615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:09.093920 17615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:09.100538 17615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:44821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:09.102027 17615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:09.103272 17615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:09.113067 17623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.113054 17620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.113329 17615 server_base.cc:1034] running on GCE node
W20260811 18:54:09.113054 17621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.114352 17615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:09.115701 17615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:09.116936 17615 hybrid_clock.cc:648] HybridClock initialized: now 1786474449116881 us; error 78 us; skew 500 ppm
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:09.120543 17615 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:09.121585 17615 webserver.cc:460] Webserver started at http://127.27.115.254:41927/ using document root <none> and password file <none>
I20260811 18:54:09.122265 17615 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:09.122341 17615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:09.122529 17615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:09.124904 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "68a48ee113af43509ced438204faf8e8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "32b2d3c3c8caea81060d426b082cc174"
server_key_iv: "66ce7b5b4c25a28a79a0bd5b78890dba"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.125730 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "68a48ee113af43509ced438204faf8e8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "32b2d3c3c8caea81060d426b082cc174"
server_key_iv: "66ce7b5b4c25a28a79a0bd5b78890dba"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.131088 17615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:54:09.134404 17630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.135984 17615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:54:09.136122 17615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "68a48ee113af43509ced438204faf8e8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "32b2d3c3c8caea81060d426b082cc174"
server_key_iv: "66ce7b5b4c25a28a79a0bd5b78890dba"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.136243 17615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:09.153492 17615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:09.157353 17615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:09.157557 17615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:09.166066 17615 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:44821
I20260811 18:54:09.166280 17684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:44821 every 8 connection(s)
I20260811 18:54:09.167485 17615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:09.171082 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:09.176138 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17615
I20260811 18:54:09.176374 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:09.176755 28111 external_mini_cluster.cc:1428] Setting key 1898f9e9e2e0c0ab2c2768412206eb5e
I20260811 18:54:09.179203 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: Bootstrap starting.
I20260811 18:54:09.181814 17685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:09.182711 17685 log.cc:824] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:09.185305 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: No bootstrap required, opened a new log
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:09.190300 17685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } }
I20260811 18:54:09.190680 17685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:09.190768 17685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a48ee113af43509ced438204faf8e8, State: Initialized, Role: FOLLOWER
I20260811 18:54:09.191255 17685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } }
I20260811 18:54:09.191381 17685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:09.191469 17685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:09.191570 17685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:09.192605 17685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } }
I20260811 18:54:09.192979 17685 leader_election.cc:302] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68a48ee113af43509ced438204faf8e8; no voters:
I20260811 18:54:09.193360 17685 leader_election.cc:288] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:09.193446 17690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:09.193653 17690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [term 1 LEADER]: Becoming Leader. State: Replica: 68a48ee113af43509ced438204faf8e8, State: Running, Role: LEADER
I20260811 18:54:09.193975 17690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } }
I20260811 18:54:09.194834 17685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:09.194948 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.178736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.27.115.254:44821)
0811 18:54:09.179463 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:09.179482 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:09.179526 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:54:09.180485 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:09.180504 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:09.181028 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:09.181501 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:09.182517 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.183871 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.185029 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.185821 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.186019 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.189329 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:09.189336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:09.189357 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:09.189397 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:09.191665 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.192186 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.192193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.192199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.192338 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.192648 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.192652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.192654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.193189 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:09.193419 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:09.193691 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 18:54:09.194050 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":183,"threads_started":1}
I20260811 18:54:09.195272 17690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68a48ee113af43509ced438204faf8e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } } }
I20260811 18:54:09.195389 17690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:09.195758 17690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68a48ee113af43509ced438204faf8e8. Latest consensus state: current_term: 1 leader_uuid: "68a48ee113af43509ced438204faf8e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a48ee113af43509ced438204faf8e8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 44821 } } }
I20260811 18:54:09.195858 17690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:09.195984 17695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:09.199405 17695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:09.205294 17695 catalog_manager.cc:1347] Generated new cluster ID: 8fcc1ba928544546a417dea92ab3a068
I20260811 18:54:09.205390 17695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:09.220386 17695 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:09.221493 17695 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:09.232446 17695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: Generated new TSK 0
I20260811 18:54:09.233426 17695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
Entry for principal kudu/127.27.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab.
I20260811 18:54:09.271142 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:09.397042 17714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:09.397385 17714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:09.397471 17714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:09.401993 17714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:09.402082 17714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:09.402225 17714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:09.408043 17714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.193.keytab
--principal=kudu/127.27.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:09.409220 17714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:09.410457 17714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:54:09.422109 17714 server_base.cc:1034] running on GCE node
W20260811 18:54:09.423357 17719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.423295 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.423332 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.424052 17714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:09.424844 17714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:09.426088 17714 hybrid_clock.cc:648] HybridClock initialized: now 1786474449426048 us; error 59 us; skew 500 ppm
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:09.429629 17714 init.cc:377] Logged in from keytab as kudu/127.27.115.193@KRBTEST.COM (short username kudu)
I20260811 18:54:09.431088 17714 webserver.cc:460] Webserver started at http://127.27.115.193:44915/ using document root <none> and password file <none>
I20260811 18:54:09.431792 17714 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:09.431900 17714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:09.432185 17714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:09.434715 17714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "9c21ecf011074a278dcc546ba1a0c70b"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "a042440e5418f68249a6ed7fd1f4208d"
server_key_iv: "8e1ad388a70f06be243314a4c196a858"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.435393 17714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "9c21ecf011074a278dcc546ba1a0c70b"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "a042440e5418f68249a6ed7fd1f4208d"
server_key_iv: "8e1ad388a70f06be243314a4c196a858"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.439769 17714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:09.442698 17731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.444206 17714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:09.444401 17714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "9c21ecf011074a278dcc546ba1a0c70b"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "a042440e5418f68249a6ed7fd1f4208d"
server_key_iv: "8e1ad388a70f06be243314a4c196a858"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.444545 17714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:09.462217 17714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:09.466383 17714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:09.466622 17714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:09.467299 17714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:09.468210 17714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:09.468292 17714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.468364 17714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:09.468416 17714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.477772 17714 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:42557
I20260811 18:54:09.477799 17852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:42557 every 8 connection(s)
I20260811 18:54:09.478852 17714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:09.480387 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17714
I20260811 18:54:09.480536 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:09.480878 28111 external_mini_cluster.cc:1428] Setting key 8a686e247e32dca8638cc755fbde0aa7
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.193@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.194@KRBTEST.COM" created.
I20260811 18:54:09.496178 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.480978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:50173 (local address 127.27.115.254:44821)
0811 18:54:09.481135 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:09.481140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:09.482378 (+ 1238us) server_negotiation.cc:407] Connection header received
0811 18:54:09.484400 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:09.484404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:09.484454 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:09.484535 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:09.486078 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.486981 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.488302 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.488726 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.488873 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.491713 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:09.491719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:09.491721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:09.491759 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:09.493365 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.494062 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.494065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.494066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.494227 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.494651 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.494658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.494663 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.494945 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:09.495079 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:09.495784 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 18:54:09.495943 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:54:09.497255 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.481564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44821 (local address 127.27.115.193:50173)
0811 18:54:09.482187 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:09.482268 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:09.484117 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:09.484785 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:09.484798 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:09.485329 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:09.485877 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.485894 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.487137 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.487140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.488095 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.488138 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.488909 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.488912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.489070 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.489756 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:09.489780 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:09.491460 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:09.493572 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.493583 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.493604 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.493951 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.494318 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.494320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.494321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.494475 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.495140 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:09.495146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:09.495487 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 18:54:09.495729 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 18:54:09.496202 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":164,"threads_started":1}
I20260811 18:54:09.498747 17853 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44821
I20260811 18:54:09.499070 17853 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:09.499742 17853 heartbeater.cc:511] Master 127.27.115.254:44821 requested a full tablet report, sending...
I20260811 18:54:09.501789 17647 ts_manager.cc:194] Registered new tserver with Master: 9c21ecf011074a278dcc546ba1a0c70b (127.27.115.193:42557)
I20260811 18:54:09.503679 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.193@KRBTEST.COM'} at 127.27.115.193:50173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
Entry for principal kudu/127.27.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab.
I20260811 18:54:09.517333 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:09.636695 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:09.637024 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:09.637112 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:09.641346 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:09.641444 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:09.641536 17860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:09.647323 17860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.194.keytab
--principal=kudu/127.27.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:09.648567 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:09.649703 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:09.660211 17865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.660373 17860 server_base.cc:1034] running on GCE node
W20260811 18:54:09.660244 17866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.660588 17869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.661064 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:09.661707 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:09.662961 17860 hybrid_clock.cc:648] HybridClock initialized: now 1786474449662919 us; error 63 us; skew 500 ppm
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:09.666366 17860 init.cc:377] Logged in from keytab as kudu/127.27.115.194@KRBTEST.COM (short username kudu)
I20260811 18:54:09.667361 17860 webserver.cc:460] Webserver started at http://127.27.115.194:44101/ using document root <none> and password file <none>
I20260811 18:54:09.668010 17860 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:09.668114 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:09.668373 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:09.670639 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "43995fb6e3014b69b4b895cdb2ae1ab1"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "ecaf997dffaa8d0b38d3a78779842fe9"
server_key_iv: "b62ed456dcf4fe8c4ec488cf85bb1736"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.671293 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "43995fb6e3014b69b4b895cdb2ae1ab1"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "ecaf997dffaa8d0b38d3a78779842fe9"
server_key_iv: "b62ed456dcf4fe8c4ec488cf85bb1736"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.675815 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:09.679147 17878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.680776 17860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:09.680967 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "43995fb6e3014b69b4b895cdb2ae1ab1"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "ecaf997dffaa8d0b38d3a78779842fe9"
server_key_iv: "b62ed456dcf4fe8c4ec488cf85bb1736"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.681120 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:09.715119 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:09.718411 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:09.718585 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:09.719194 17860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:09.720075 17860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:09.720126 17860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.720171 17860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:09.720187 17860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.730204 17860 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:40651
I20260811 18:54:09.730232 17999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:40651 every 8 connection(s)
I20260811 18:54:09.731475 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:09.736280 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 17860
I20260811 18:54:09.736438 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:09.736707 28111 external_mini_cluster.cc:1428] Setting key c685b357d580a72112f98dad53ae05c3
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.194@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:09.747216 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.733605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:56885 (local address 127.27.115.254:44821)
0811 18:54:09.733744 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:09.733748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:09.734759 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 18:54:09.736070 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:09.736074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:09.736139 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:09.736250 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:09.737582 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.738119 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.739341 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.739582 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.739699 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.742908 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:09.742912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:09.742915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:09.742952 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:09.744750 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.745382 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.745386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.745388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.745483 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.745878 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.745881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.745883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.746116 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:09.746277 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:09.746882 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 18:54:09.747041 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.27.115.195@KRBTEST.COM; defaulting to no policy
I20260811 18:54:09.748241 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.734022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44821 (local address 127.27.115.194:56885)
0811 18:54:09.734600 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:09.734645 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:09.735849 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:09.736455 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:09.736462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:09.736928 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:09.737430 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.737444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.738291 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.738295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.739200 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.739210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.739786 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.739794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.739955 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.740678 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:09.740708 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:09.742681 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:09.744902 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.744911 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.744931 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.745255 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.745586 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.745589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.745591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.745765 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.746269 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:09.746278 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:09.746595 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 18:54:09.746833 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 18:54:09.747222 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":158,"threads_started":1}
I20260811 18:54:09.749975 18000 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44821
I20260811 18:54:09.750301 18000 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.195@KRBTEST.COM" created.
I20260811 18:54:09.751011 18000 heartbeater.cc:511] Master 127.27.115.254:44821 requested a full tablet report, sending...
I20260811 18:54:09.752394 17647 ts_manager.cc:194] Registered new tserver with Master: 43995fb6e3014b69b4b895cdb2ae1ab1 (127.27.115.194:40651)
I20260811 18:54:09.753294 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.194@KRBTEST.COM'} at 127.27.115.194:56885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
Entry for principal kudu/127.27.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab.
I20260811 18:54:09.770675 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:09.895085 18006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:09.895465 18006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:09.895536 18006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:09.899995 18006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:09.900082 18006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:09.900146 18006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:09.905997 18006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.195.keytab
--principal=kudu/127.27.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:44821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:09.907296 18006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:09.908464 18006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:09.919356 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.919365 18014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:09.919365 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:09.919448 18006 server_base.cc:1034] running on GCE node
I20260811 18:54:09.920040 18006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:09.920807 18006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:09.921996 18006 hybrid_clock.cc:648] HybridClock initialized: now 1786474449921961 us; error 47 us; skew 500 ppm
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:09.925170 18006 init.cc:377] Logged in from keytab as kudu/127.27.115.195@KRBTEST.COM (short username kudu)
I20260811 18:54:09.926249 18006 webserver.cc:460] Webserver started at http://127.27.115.195:37909/ using document root <none> and password file <none>
I20260811 18:54:09.926854 18006 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:09.926918 18006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:09.927145 18006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:09.929752 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "4a5eb853c5044c249792db0c130cffd8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "0f8110c77846aabef5f25d32273d2bc5"
server_key_iv: "40a6d4da666a3709129a4cb15a20db11"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.930405 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "4a5eb853c5044c249792db0c130cffd8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "0f8110c77846aabef5f25d32273d2bc5"
server_key_iv: "40a6d4da666a3709129a4cb15a20db11"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.934504 18006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:54:09.937177 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.938747 18006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:09.938891 18006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "4a5eb853c5044c249792db0c130cffd8"
format_stamp: "Formatted at 2026-08-11 18:54:09 on dist-test-slave-9jjr"
server_key: "0f8110c77846aabef5f25d32273d2bc5"
server_key_iv: "40a6d4da666a3709129a4cb15a20db11"
server_key_version: "encryptionkey@0"
I20260811 18:54:09.939011 18006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:09.957266 18006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:09.960918 18006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:09.961160 18006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:09.961827 18006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:09.962881 18006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:09.962934 18006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.963001 18006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:09.963042 18006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:09.972662 18006 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:34295
I20260811 18:54:09.972687 18151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:34295 every 8 connection(s)
I20260811 18:54:09.973665 18006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:09.979147 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18006
I20260811 18:54:09.979310 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:09.979666 28111 external_mini_cluster.cc:1428] Setting key 25ab3aed526c8094dfd877180d1701ef
Aug 11 18:54:09 dist-test-slave-9jjr krb5kdc[17602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474449, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.195@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:09.990389 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.975691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:36961 (local address 127.27.115.254:44821)
0811 18:54:09.975866 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:09.975870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:09.976759 (+ 889us) server_negotiation.cc:407] Connection header received
0811 18:54:09.978231 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:09.978235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:09.978302 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:09.978385 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:09.980139 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.980833 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.982369 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.982612 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.982705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.985932 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:09.985937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:09.985946 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:09.985981 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:09.987543 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.988374 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.988377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.988378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.988479 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:09.988933 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:09.988936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:09.988937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:09.989201 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:09.989295 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:09.989942 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 18:54:09.990230 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:54:09.991535 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:09.976091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:44821 (local address 127.27.115.195:36961)
0811 18:54:09.976578 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:09.976632 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:09.977942 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:09.978590 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:09.978597 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:09.979137 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:09.979891 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.979914 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.981003 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.981009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.982250 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:09.982262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:09.982786 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:09.982789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:09.982956 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:09.983669 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:09.983687 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:09.985584 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:09.987735 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.987742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.987760 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.988197 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.988617 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:09.988620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:09.988621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:09.988821 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:09.989309 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:09.989315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:09.989622 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 18:54:09.989822 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 18:54:09.990370 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260811 18:54:09.993366 18152 heartbeater.cc:347] Connected to a master server at 127.27.115.254:44821
I20260811 18:54:09.993796 18152 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:09.994664 18152 heartbeater.cc:511] Master 127.27.115.254:44821 requested a full tablet report, sending...
I20260811 18:54:09.996019 17647 ts_manager.cc:194] Registered new tserver with Master: 4a5eb853c5044c249792db0c130cffd8 (127.27.115.195:34295)
I20260811 18:54:09.996690 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.115.195@KRBTEST.COM'} at 127.27.115.195:36961
I20260811 18:54:10.007295 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:10.018992 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.009664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.27.115.254:44821)
0811 18:54:10.009893 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.009897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.010043 (+ 146us) server_negotiation.cc:407] Connection header received
0811 18:54:10.010279 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.010287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.010336 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.010438 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:10.011278 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.011830 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.012738 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.012987 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.013125 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.014812 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:10.014816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:10.014818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:10.014851 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:10.016698 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:10.017331 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:10.017334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:10.017336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:10.017446 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:10.017796 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:10.017801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:10.017802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:10.018086 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:10.018311 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.018632 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.018787 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 18:54:10.023524 17647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:10.027213 17647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:10.044291 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.037649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:34295 (local address 127.0.0.1:57292)
0811 18:54:10.038382 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.038442 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.038592 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.039200 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.039206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.039232 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:10.039399 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.039407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.040851 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.040856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.042301 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.042313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.043598 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.043602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.043734 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.043783 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.043921 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.044076 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":104,"threads_started":1}
I20260811 18:54:10.044341 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.038059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.27.115.195:34295)
0811 18:54:10.038571 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.038584 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.038609 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:54:10.038807 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.038814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.039031 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.039199 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:10.039551 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.040719 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.042461 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.043456 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.043603 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.043779 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.043974 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.044104 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":208,"threads_started":1}
I20260811 18:54:10.044341 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.037649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:42557 (local address 127.0.0.1:46456)
0811 18:54:10.038820 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.038839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.038941 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.039677 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.039680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.039698 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:10.039786 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.039792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.041311 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.041315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.042482 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.042491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.043866 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.043870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.044004 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.044030 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.044119 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.044200 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":23,"thread_start_us":130,"threads_started":1}
I20260811 18:54:10.044858 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.038501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:40651 (local address 127.0.0.1:35822)
0811 18:54:10.039469 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.039483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.039564 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.040294 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.040297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.040316 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:10.040398 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.040404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.041765 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.041768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.042914 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.042925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.044405 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.044410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.044508 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.044531 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.044612 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.044690 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":103,"threads_started":1}
I20260811 18:54:10.045336 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.037797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.27.115.193:42557)
0811 18:54:10.038821 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.038827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.039045 (+ 218us) server_negotiation.cc:407] Connection header received
0811 18:54:10.039161 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.039166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.039334 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.039525 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:10.039921 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.041162 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.042636 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.043577 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.043704 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.043814 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.045049 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.045187 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":101,"threads_started":1}
I20260811 18:54:10.048121 17784 tablet_service.cc:1467] Processing CreateTablet for tablet 417ed66a4c4a4deea961674dc81135b8 (DEFAULT_TABLE table=test-table [id=669e9b202d9f45f08d1112be7f9cbbac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:10.048111 18075 tablet_service.cc:1467] Processing CreateTablet for tablet 417ed66a4c4a4deea961674dc81135b8 (DEFAULT_TABLE table=test-table [id=669e9b202d9f45f08d1112be7f9cbbac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:10.049445 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417ed66a4c4a4deea961674dc81135b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:10.049445 18075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417ed66a4c4a4deea961674dc81135b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:10.049479 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.038731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.27.115.194:40651)
0811 18:54:10.039800 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.039810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.039828 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:10.039931 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.039935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.040108 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.040306 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:10.040545 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.041565 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.043050 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.044067 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.044192 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.044327 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.049082 (+ 4755us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.049318 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":126,"threads_started":1}
I20260811 18:54:10.052949 17929 tablet_service.cc:1467] Processing CreateTablet for tablet 417ed66a4c4a4deea961674dc81135b8 (DEFAULT_TABLE table=test-table [id=669e9b202d9f45f08d1112be7f9cbbac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:10.054268 17929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417ed66a4c4a4deea961674dc81135b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:10.057709 18171 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: Bootstrap starting.
I20260811 18:54:10.060616 18171 tablet_bootstrap.cc:654] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:10.061149 18172 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Bootstrap starting.
I20260811 18:54:10.062852 18173 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Bootstrap starting.
I20260811 18:54:10.063390 18171 log.cc:824] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:10.063953 18172 tablet_bootstrap.cc:654] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:10.065305 18172 log.cc:824] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:10.065696 18173 tablet_bootstrap.cc:654] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:10.066440 18171 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: No bootstrap required, opened a new log
I20260811 18:54:10.066733 18171 ts_tablet_manager.cc:1397] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:54:10.067196 18173 log.cc:824] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:10.067552 18172 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: No bootstrap required, opened a new log
I20260811 18:54:10.067790 18172 ts_tablet_manager.cc:1397] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 18:54:10.069684 18173 tablet_bootstrap.cc:492] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: No bootstrap required, opened a new log
I20260811 18:54:10.069924 18173 ts_tablet_manager.cc:1397] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260811 18:54:10.071274 18171 raft_consensus.cc:348] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.071652 18171 raft_consensus.cc:374] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:10.071715 18171 raft_consensus.cc:729] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c21ecf011074a278dcc546ba1a0c70b, State: Initialized, Role: FOLLOWER
I20260811 18:54:10.072257 18171 consensus_queue.cc:260] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.072150 18172 raft_consensus.cc:348] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.072422 18172 raft_consensus.cc:374] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:10.072494 18172 raft_consensus.cc:729] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5eb853c5044c249792db0c130cffd8, State: Initialized, Role: FOLLOWER
I20260811 18:54:10.072957 18172 consensus_queue.cc:260] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.073154 17853 heartbeater.cc:503] Master 127.27.115.254:44821 was elected leader, sending a full tablet report...
I20260811 18:54:10.073506 18171 ts_tablet_manager.cc:1428] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:54:10.073976 18172 ts_tablet_manager.cc:1428] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 18:54:10.074391 18152 heartbeater.cc:503] Master 127.27.115.254:44821 was elected leader, sending a full tablet report...
I20260811 18:54:10.074210 18173 raft_consensus.cc:348] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.074486 18173 raft_consensus.cc:374] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:10.074542 18173 raft_consensus.cc:729] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43995fb6e3014b69b4b895cdb2ae1ab1, State: Initialized, Role: FOLLOWER
I20260811 18:54:10.075027 18173 consensus_queue.cc:260] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.075798 18000 heartbeater.cc:503] Master 127.27.115.254:44821 was elected leader, sending a full tablet report...
I20260811 18:54:10.076074 18173 ts_tablet_manager.cc:1428] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 18:54:10.078713 18179 raft_consensus.cc:482] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:54:10.078888 18179 raft_consensus.cc:504] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.080245 18179 leader_election.cc:288] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c21ecf011074a278dcc546ba1a0c70b (127.27.115.193:42557), 4a5eb853c5044c249792db0c130cffd8 (127.27.115.195:34295)
I20260811 18:54:10.084697 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.080655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:41005 (local address 127.27.115.195:34295)
0811 18:54:10.080831 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.080837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.080952 (+ 115us) server_negotiation.cc:407] Connection header received
0811 18:54:10.081106 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.081108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.081152 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.081219 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:10.081575 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.082233 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.083602 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.084037 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.084148 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.084233 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.084463 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.084551 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:54:10.084697 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.080510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:42557 (local address 127.27.115.194:33013)
0811 18:54:10.080881 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.080898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.081013 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.081458 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.081461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.081479 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:10.081606 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.081613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.082450 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.082453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.083442 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.083455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.084229 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.084233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.084352 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.084379 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.084482 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.084558 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":67,"spinlock_wait_cycles":24064}
I20260811 18:54:10.084739 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.080745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:33013 (local address 127.27.115.193:42557)
0811 18:54:10.081219 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.081225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.081239 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:54:10.081290 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.081292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.081340 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.081423 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:10.081708 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.082330 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.083603 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.084064 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.084223 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.084280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.084519 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.084599 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
I20260811 18:54:10.084802 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.080510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:34295 (local address 127.27.115.194:41005)
0811 18:54:10.080881 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.080898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.081013 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.081261 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.081265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.081285 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:10.081448 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.081456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.082356 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.082363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.083442 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.083455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.084183 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.084187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.084318 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.084345 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.084419 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.084533 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":91,"spinlock_wait_cycles":9472,"thread_start_us":163,"threads_started":1}
I20260811 18:54:10.085649 17805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417ed66a4c4a4deea961674dc81135b8" candidate_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c21ecf011074a278dcc546ba1a0c70b" is_pre_election: true
I20260811 18:54:10.085968 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417ed66a4c4a4deea961674dc81135b8" candidate_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5eb853c5044c249792db0c130cffd8" is_pre_election: true
I20260811 18:54:10.086026 17805 raft_consensus.cc:2454] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43995fb6e3014b69b4b895cdb2ae1ab1 in term 0.
I20260811 18:54:10.086309 18097 raft_consensus.cc:2454] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43995fb6e3014b69b4b895cdb2ae1ab1 in term 0.
I20260811 18:54:10.086768 17881 leader_election.cc:302] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 43995fb6e3014b69b4b895cdb2ae1ab1, 4a5eb853c5044c249792db0c130cffd8, 9c21ecf011074a278dcc546ba1a0c70b; no voters:
I20260811 18:54:10.087132 18179 raft_consensus.cc:2781] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:10.087221 18179 raft_consensus.cc:482] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:10.087252 18179 raft_consensus.cc:3037] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:10.088572 18179 raft_consensus.cc:504] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.088984 18179 leader_election.cc:288] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 9c21ecf011074a278dcc546ba1a0c70b (127.27.115.193:42557), 4a5eb853c5044c249792db0c130cffd8 (127.27.115.195:34295)
I20260811 18:54:10.089459 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417ed66a4c4a4deea961674dc81135b8" candidate_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5eb853c5044c249792db0c130cffd8"
I20260811 18:54:10.089622 18097 raft_consensus.cc:3037] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:10.089645 17805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417ed66a4c4a4deea961674dc81135b8" candidate_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c21ecf011074a278dcc546ba1a0c70b"
I20260811 18:54:10.089818 17805 raft_consensus.cc:3037] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:10.091149 18097 raft_consensus.cc:2454] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43995fb6e3014b69b4b895cdb2ae1ab1 in term 1.
I20260811 18:54:10.091162 17805 raft_consensus.cc:2454] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43995fb6e3014b69b4b895cdb2ae1ab1 in term 1.
I20260811 18:54:10.091527 17879 leader_election.cc:302] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43995fb6e3014b69b4b895cdb2ae1ab1, 4a5eb853c5044c249792db0c130cffd8; no voters:
I20260811 18:54:10.091804 18179 raft_consensus.cc:2781] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:10.092105 18179 raft_consensus.cc:686] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 1 LEADER]: Becoming Leader. State: Replica: 43995fb6e3014b69b4b895cdb2ae1ab1, State: Running, Role: LEADER
I20260811 18:54:10.092553 18179 consensus_queue.cc:237] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } }
I20260811 18:54:10.096280 17645 catalog_manager.cc:5495] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43995fb6e3014b69b4b895cdb2ae1ab1 (127.27.115.194). New cstate: current_term: 1 leader_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43995fb6e3014b69b4b895cdb2ae1ab1" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 40651 } health_report { overall_health: HEALTHY } } }
I20260811 18:54:10.112591 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.108530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.27.115.194:40651)
0811 18:54:10.108677 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.108681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.108696 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:10.108787 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.108789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.108850 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.109035 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:10.109333 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.109875 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.110994 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.111219 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.111319 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.111687 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:10.112164 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:10.112273 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.112392 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.112462 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:54:10.120199 18097 raft_consensus.cc:1261] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 1 FOLLOWER]: Refusing update from remote peer 43995fb6e3014b69b4b895cdb2ae1ab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:10.120199 17805 raft_consensus.cc:1261] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 1 FOLLOWER]: Refusing update from remote peer 43995fb6e3014b69b4b895cdb2ae1ab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:10.121047 18181 consensus_queue.cc:1035] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5eb853c5044c249792db0c130cffd8" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 34295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:10.121284 18179 consensus_queue.cc:1035] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c21ecf011074a278dcc546ba1a0c70b" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:10.131057 18186 mvcc.cc:205] Tried to move back new op lower bound from 7317399347681894400 to 7317399347582136320. Current Snapshot: MvccSnapshot[applied={T|T < 7317399347681894400}]
I20260811 18:54:10.131932 18187 mvcc.cc:205] Tried to move back new op lower bound from 7317399347681894400 to 7317399347582136320. Current Snapshot: MvccSnapshot[applied={T|T < 7317399347681894400}]
I20260811 18:54:10.143510 17645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:10.143784 17645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:10.146845 17645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68a48ee113af43509ced438204faf8e8: Sending DeleteTablet for 3 replicas of tablet 417ed66a4c4a4deea961674dc81135b8
I20260811 18:54:10.147838 17929 tablet_service.cc:1514] Processing DeleteTablet for tablet 417ed66a4c4a4deea961674dc81135b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:10 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:35822
I20260811 18:54:10.147879 18075 tablet_service.cc:1514] Processing DeleteTablet for tablet 417ed66a4c4a4deea961674dc81135b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:10 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:57292
I20260811 18:54:10.148581 18195 tablet_replica.cc:330] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: stopping tablet replica
I20260811 18:54:10.148545 17784 tablet_service.cc:1514] Processing DeleteTablet for tablet 417ed66a4c4a4deea961674dc81135b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:10 UTC) from {username='kudu', principal='kudu/127.27.115.254@KRBTEST.COM'} at 127.0.0.1:46456
I20260811 18:54:10.149063 18195 raft_consensus.cc:2229] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:10.149212 18197 tablet_replica.cc:330] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b: stopping tablet replica
I20260811 18:54:10.149647 18197 raft_consensus.cc:2229] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:10.149695 18195 raft_consensus.cc:2258] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:10.150043 18197 raft_consensus.cc:2258] T 417ed66a4c4a4deea961674dc81135b8 P 9c21ecf011074a278dcc546ba1a0c70b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:10.150269 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17714
I20260811 18:54:10.152864 18195 ts_tablet_manager.cc:1905] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:10.156348 18195 ts_tablet_manager.cc:1918] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:10.156494 18195 log.cc:1196] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/417ed66a4c4a4deea961674dc81135b8
I20260811 18:54:10.156889 18195 ts_tablet_manager.cc:1939] T 417ed66a4c4a4deea961674dc81135b8 P 4a5eb853c5044c249792db0c130cffd8: Deleting consensus metadata
I20260811 18:54:10.157317 18196 tablet_replica.cc:330] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: stopping tablet replica
I20260811 18:54:10.157807 18196 raft_consensus.cc:2229] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:54:10.158489 17631 catalog_manager.cc:4841] TS 4a5eb853c5044c249792db0c130cffd8 (127.27.115.195:34295): tablet 417ed66a4c4a4deea961674dc81135b8 (table test-table [id=669e9b202d9f45f08d1112be7f9cbbac]) successfully deleted
I20260811 18:54:10.158943 18196 raft_consensus.cc:2258] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:10.160887 18196 ts_tablet_manager.cc:1905] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:54:10.164734 17633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:42557 (error 108)
I20260811 18:54:10.164852 18196 ts_tablet_manager.cc:1918] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:10.164947 18196 log.cc:1196] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/417ed66a4c4a4deea961674dc81135b8
I20260811 18:54:10.165318 18196 ts_tablet_manager.cc:1939] T 417ed66a4c4a4deea961674dc81135b8 P 43995fb6e3014b69b4b895cdb2ae1ab1: Deleting consensus metadata
I20260811 18:54:10.166186 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.165744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:42557 (local address 127.0.0.1:46470)
0811 18:54:10.165997 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.166066 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260811 18:54:10.166373 17633 catalog_manager.cc:4584] TS 9c21ecf011074a278dcc546ba1a0c70b (127.27.115.193:42557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 417ed66a4c4a4deea961674dc81135b8: Network error: Client connection negotiation failed: client connection to 127.27.115.193:42557: connect: Connection refused (error 111)
I20260811 18:54:10.166752 17634 catalog_manager.cc:4841] TS 43995fb6e3014b69b4b895cdb2ae1ab1 (127.27.115.194:40651): tablet 417ed66a4c4a4deea961674dc81135b8 (table test-table [id=669e9b202d9f45f08d1112be7f9cbbac]) successfully deleted
I20260811 18:54:10.167152 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17860
I20260811 18:54:10.179083 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18006
I20260811 18:54:10.191094 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 17615
2026-08-11T18:54:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2499 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T18:54:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:10Z Disabled control of system clock
I20260811 18:54:10.228228 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36639
--webserver_interface=127.27.115.254
--webserver_port=0
--builtin_ntp_servers=127.27.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:36639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:10.349860 18203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:10.350248 18203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:10.350353 18203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:10.355067 18203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:10.355173 18203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:10.355191 18203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:10.355212 18203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:10.355228 18203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:10.360566 18203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:36639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:10.361866 18203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:10.363420 18203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:10.372844 18212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:10.372960 18203 server_base.cc:1034] running on GCE node
W20260811 18:54:10.372870 18208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:10.372849 18209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:10.373967 18203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:10.375398 18203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:10.376647 18203 hybrid_clock.cc:648] HybridClock initialized: now 1786474450376620 us; error 38 us; skew 500 ppm
I20260811 18:54:10.383490 18203 webserver.cc:460] Webserver started at http://127.27.115.254:35475/ using document root <none> and password file <none>
I20260811 18:54:10.384279 18203 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:10.384364 18203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:10.384644 18203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:10.387300 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "f2dc2a07a3114ee485b754cd46a058a6"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "db57e7c40080996a4f043f0130839700"
server_key_iv: "8bb27e5267b1ec0d8df4f15e014bc440"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.387989 18203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "f2dc2a07a3114ee485b754cd46a058a6"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "db57e7c40080996a4f043f0130839700"
server_key_iv: "8bb27e5267b1ec0d8df4f15e014bc440"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.392369 18203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:10.395237 18220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.396627 18203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:10.396780 18203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "f2dc2a07a3114ee485b754cd46a058a6"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "db57e7c40080996a4f043f0130839700"
server_key_iv: "8bb27e5267b1ec0d8df4f15e014bc440"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.396922 18203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:10.448469 18203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:10.449301 18203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:10.449522 18203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:10.457309 18203 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:36639
I20260811 18:54:10.457324 18277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:36639 every 8 connection(s)
I20260811 18:54:10.458408 18203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:10.461743 18278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:10.467161 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18203
I20260811 18:54:10.467324 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:10.467620 28111 external_mini_cluster.cc:1428] Setting key f17dcdee2aaab340652e152b1aa9bd2a
I20260811 18:54:10.468147 18278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: Bootstrap starting.
I20260811 18:54:10.471448 18278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:10.472353 18278 log.cc:824] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:10.474790 18278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: No bootstrap required, opened a new log
I20260811 18:54:10.478068 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.469221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.27.115.254:36639)
0811 18:54:10.470202 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.470224 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.470272 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:54:10.471056 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.471086 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.471538 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.471975 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:10.472516 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.473546 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.474563 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.475134 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.475317 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.476249 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:10.476254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:10.476269 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:10.476682 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:10.476744 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:10.476766 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:10.476880 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.477001 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.477291 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":165,"threads_started":1}
I20260811 18:54:10.478842 18278 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } }
I20260811 18:54:10.479125 18278 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:10.479177 18278 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2dc2a07a3114ee485b754cd46a058a6, State: Initialized, Role: FOLLOWER
I20260811 18:54:10.479646 18278 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } }
I20260811 18:54:10.479768 18278 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:10.479810 18278 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:10.479876 18278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:10.481020 18278 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } }
I20260811 18:54:10.481364 18278 leader_election.cc:302] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2dc2a07a3114ee485b754cd46a058a6; no voters:
I20260811 18:54:10.481676 18278 leader_election.cc:288] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:10.481854 18284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:10.482108 18284 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [term 1 LEADER]: Becoming Leader. State: Replica: f2dc2a07a3114ee485b754cd46a058a6, State: Running, Role: LEADER
I20260811 18:54:10.482503 18284 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } }
I20260811 18:54:10.483155 18278 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:10.484401 18287 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2dc2a07a3114ee485b754cd46a058a6. Latest consensus state: current_term: 1 leader_uuid: "f2dc2a07a3114ee485b754cd46a058a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } } }
I20260811 18:54:10.484558 18287 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:10.484884 18285 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2dc2a07a3114ee485b754cd46a058a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc2a07a3114ee485b754cd46a058a6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36639 } } }
I20260811 18:54:10.484989 18285 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:10.485364 18295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:10.489059 18295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:10.495766 18295 catalog_manager.cc:1347] Generated new cluster ID: ee7f91c632cd4ac69caf0b0a9bfaf174
I20260811 18:54:10.495867 18295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:10.505754 18295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:10.507081 18295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:10.516651 18295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: Generated new TSK 0
I20260811 18:54:10.517545 18295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:10.528890 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.193:0
--local_ip_for_outbound_sockets=127.27.115.193
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--builtin_ntp_servers=127.27.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:54:10.648133 18307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:10.648412 18307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:10.648452 18307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:10.652637 18307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:10.652721 18307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:10.652786 18307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.193
I20260811 18:54:10.658903 18307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.115.193
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.193
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:10.660326 18307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:10.662240 18307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:10.674193 18315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:10.674342 18307 server_base.cc:1034] running on GCE node
W20260811 18:54:10.674198 18313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:10.674518 18317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:10.674991 18307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:10.675712 18307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:10.676923 18307 hybrid_clock.cc:648] HybridClock initialized: now 1786474450676903 us; error 32 us; skew 500 ppm
I20260811 18:54:10.679150 18307 webserver.cc:460] Webserver started at http://127.27.115.193:37067/ using document root <none> and password file <none>
I20260811 18:54:10.679876 18307 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:10.679975 18307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:10.680222 18307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:10.682508 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data/instance:
uuid: "07f5ee785c854dceac306ee68f929f2e"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "5f367892fa06da942f3c33ad98c2f049"
server_key_iv: "26b046c65b27e699a046d51ac44d47fd"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.683132 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance:
uuid: "07f5ee785c854dceac306ee68f929f2e"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "5f367892fa06da942f3c33ad98c2f049"
server_key_iv: "26b046c65b27e699a046d51ac44d47fd"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.687093 18307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:10.689706 18323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.691007 18307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:54:10.691154 18307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal
uuid: "07f5ee785c854dceac306ee68f929f2e"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "5f367892fa06da942f3c33ad98c2f049"
server_key_iv: "26b046c65b27e699a046d51ac44d47fd"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.691305 18307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:10.720487 18307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:10.721377 18307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:10.721616 18307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:10.722419 18307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:10.723578 18307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:10.723666 18307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.723749 18307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:10.723794 18307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.736865 18307 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.193:43489
I20260811 18:54:10.736922 18456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.193:43489 every 8 connection(s)
I20260811 18:54:10.737913 18307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:10.747691 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18307
I20260811 18:54:10.747799 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:10.748178 28111 external_mini_cluster.cc:1428] Setting key 751c52b8d02cf0be05161987b2e8da63
I20260811 18:54:10.749361 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.740030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:48933 (local address 127.27.115.254:36639)
0811 18:54:10.740214 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.740218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.741605 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 18:54:10.743203 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.743206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.743280 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.743392 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:10.744930 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.745496 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.747020 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.747321 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.747447 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.748770 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:10.748774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:10.748777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:10.748793 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:10.748824 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:10.748833 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:10.748922 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.749132 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.749197 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:54:10.750464 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.740456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36639 (local address 127.27.115.193:48933)
0811 18:54:10.741332 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.741389 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.742957 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.743608 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.743631 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.743755 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:10.744756 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.744774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.745645 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.745648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.746831 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.746844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.747503 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.747507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.747696 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.748467 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:10.748468 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:10.748514 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:10.748588 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:10.748924 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:10.748938 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:10.748948 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.749091 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.749432 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":147,"threads_started":1}
I20260811 18:54:10.750641 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.194:0
--local_ip_for_outbound_sockets=127.27.115.194
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--builtin_ntp_servers=127.27.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:10.752125 18457 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36639
I20260811 18:54:10.752439 18457 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:10.753065 18457 heartbeater.cc:511] Master 127.27.115.254:36639 requested a full tablet report, sending...
I20260811 18:54:10.754949 18238 ts_manager.cc:194] Registered new tserver with Master: 07f5ee785c854dceac306ee68f929f2e (127.27.115.193:43489)
I20260811 18:54:10.756276 18238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.193:48933
W20260811 18:54:10.873602 18461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:10.873898 18461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:10.873939 18461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:10.878202 18461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:10.878325 18461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:10.878428 18461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.194
I20260811 18:54:10.883819 18461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.115.194
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.194
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:10.885180 18461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:10.886762 18461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:10.896992 18470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:10.897033 18467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:10.897012 18468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:10.897204 18461 server_base.cc:1034] running on GCE node
I20260811 18:54:10.897809 18461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:10.898721 18461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:10.899948 18461 hybrid_clock.cc:648] HybridClock initialized: now 1786474450899911 us; error 46 us; skew 500 ppm
I20260811 18:54:10.902271 18461 webserver.cc:460] Webserver started at http://127.27.115.194:35907/ using document root <none> and password file <none>
I20260811 18:54:10.902863 18461 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:10.902925 18461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:10.903115 18461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:10.905220 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data/instance:
uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "05dbb3903dfde29a88c86980f28f70ba"
server_key_iv: "1444955c60c71ce10c6b3b385ab8a1a4"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.905767 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "05dbb3903dfde29a88c86980f28f70ba"
server_key_iv: "1444955c60c71ce10c6b3b385ab8a1a4"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.909912 18461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 18:54:10.912837 18477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.914327 18461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:10.914462 18461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal
uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3"
format_stamp: "Formatted at 2026-08-11 18:54:10 on dist-test-slave-9jjr"
server_key: "05dbb3903dfde29a88c86980f28f70ba"
server_key_iv: "1444955c60c71ce10c6b3b385ab8a1a4"
server_key_version: "encryptionkey@0"
I20260811 18:54:10.914585 18461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:10.934262 18461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:10.935014 18461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:10.935204 18461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:10.935817 18461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:10.936832 18461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:10.936880 18461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.936923 18461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:10.936939 18461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:10.947274 18461 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.194:42039
I20260811 18:54:10.947321 18600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.194:42039 every 8 connection(s)
I20260811 18:54:10.948329 18461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:10.949816 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18461
I20260811 18:54:10.949960 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal/instance
I20260811 18:54:10.950364 28111 external_mini_cluster.cc:1428] Setting key 2ff199ba17d7c8b0a2e243aad8a55a90
I20260811 18:54:10.952674 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.115.195:0
--local_ip_for_outbound_sockets=127.27.115.195
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--builtin_ntp_servers=127.27.115.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:54:10.960520 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.950481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.194:56957 (local address 127.27.115.254:36639)
0811 18:54:10.950623 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:10.950628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:10.951971 (+ 1343us) server_negotiation.cc:407] Connection header received
0811 18:54:10.953962 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:10.953966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:10.954027 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:10.954110 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:10.955496 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.956048 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.957976 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.958287 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.958387 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.959966 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:10.959970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:10.959972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:10.959989 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:10.960012 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:10.960021 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:10.960107 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:10.960330 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 18:54:10.960392 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:54:10.962157 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:10.950971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36639 (local address 127.27.115.194:56957)
0811 18:54:10.951797 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:10.951857 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:10.953716 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:10.954356 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:10.954370 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:10.954513 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:10.955302 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.955326 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.956199 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.956205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.957811 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:10.957827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:10.958481 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:10.958487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:10.958761 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:10.959559 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:10.959564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:10.959634 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:10.959747 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:10.960134 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:10.960147 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:10.960157 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:54:10.960332 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:54:10.960838 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":198,"threads_started":1}
I20260811 18:54:10.963611 18602 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36639
I20260811 18:54:10.963932 18602 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:10.964790 18602 heartbeater.cc:511] Master 127.27.115.254:36639 requested a full tablet report, sending...
I20260811 18:54:10.966399 18238 ts_manager.cc:194] Registered new tserver with Master: 0b0ac0e2767d4bd0aef8faa84330a9a3 (127.27.115.194:42039)
I20260811 18:54:10.967056 18238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.194:56957
W20260811 18:54:11.076121 18605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:11.076478 18605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:11.076557 18605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:11.080823 18605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:11.080936 18605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:11.081038 18605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.115.195
I20260811 18:54:11.086488 18605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.115.195
--webserver_port=0
--tserver_master_addrs=127.27.115.254:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.115.195
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:11.087791 18605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:11.089314 18605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:11.099906 18613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:11.099906 18611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:11.099932 18610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:11.100133 18605 server_base.cc:1034] running on GCE node
I20260811 18:54:11.100665 18605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:11.101354 18605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:11.102542 18605 hybrid_clock.cc:648] HybridClock initialized: now 1786474451102524 us; error 32 us; skew 500 ppm
I20260811 18:54:11.104489 18605 webserver.cc:460] Webserver started at http://127.27.115.195:41931/ using document root <none> and password file <none>
I20260811 18:54:11.105142 18605 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:11.105240 18605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:11.105492 18605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:11.107712 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data/instance:
uuid: "f8c91e6133314e3dbbbadf1f69aa61a2"
format_stamp: "Formatted at 2026-08-11 18:54:11 on dist-test-slave-9jjr"
server_key: "2a39890f3f7259c6eeb4edcbba16a266"
server_key_iv: "e0fe70989dcff56f15e86307b2e2c3f1"
server_key_version: "encryptionkey@0"
I20260811 18:54:11.108323 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance:
uuid: "f8c91e6133314e3dbbbadf1f69aa61a2"
format_stamp: "Formatted at 2026-08-11 18:54:11 on dist-test-slave-9jjr"
server_key: "2a39890f3f7259c6eeb4edcbba16a266"
server_key_iv: "e0fe70989dcff56f15e86307b2e2c3f1"
server_key_version: "encryptionkey@0"
I20260811 18:54:11.112289 18605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:11.114956 18619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:11.116456 18605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:11.116631 18605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal
uuid: "f8c91e6133314e3dbbbadf1f69aa61a2"
format_stamp: "Formatted at 2026-08-11 18:54:11 on dist-test-slave-9jjr"
server_key: "2a39890f3f7259c6eeb4edcbba16a266"
server_key_iv: "e0fe70989dcff56f15e86307b2e2c3f1"
server_key_version: "encryptionkey@0"
I20260811 18:54:11.116771 18605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:11.132581 18605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:11.133374 18605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:11.133592 18605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:11.134326 18605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:11.135347 18605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:11.135424 18605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:11.135500 18605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:11.135545 18605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:11.146909 18605 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.195:41157
I20260811 18:54:11.146963 18739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.195:41157 every 8 connection(s)
I20260811 18:54:11.148087 18605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:11.151458 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18605
I20260811 18:54:11.151557 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:11.151933 28111 external_mini_cluster.cc:1428] Setting key 0013a325155873ecc49ec7e1903c884c
I20260811 18:54:11.159430 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.150416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.195:37525 (local address 127.27.115.254:36639)
0811 18:54:11.150577 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.150582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.152518 (+ 1936us) server_negotiation.cc:407] Connection header received
0811 18:54:11.154453 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.154456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.154516 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.154594 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:11.155560 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.156112 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.157319 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.157545 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.157661 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.158901 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:11.158905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:11.158908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:11.158929 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:11.158949 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:11.158956 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:11.159038 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.159212 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.159277 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:54:11.160317 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.150902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:36639 (local address 127.27.115.195:37525)
0811 18:54:11.152333 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.152385 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.154196 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.154792 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.154800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.154889 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:11.155418 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.155430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.156253 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.156257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.157179 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.157188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.157712 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.157716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.157893 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.158621 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:54:11.158622 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:11.158668 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:54:11.158740 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:11.159040 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:11.159052 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:11.159060 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.159200 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.159501 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":158,"threads_started":1}
I20260811 18:54:11.161554 18740 heartbeater.cc:347] Connected to a master server at 127.27.115.254:36639
I20260811 18:54:11.161842 18740 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:11.162503 18740 heartbeater.cc:511] Master 127.27.115.254:36639 requested a full tablet report, sending...
I20260811 18:54:11.163683 18238 ts_manager.cc:194] Registered new tserver with Master: f8c91e6133314e3dbbbadf1f69aa61a2 (127.27.115.195:41157)
I20260811 18:54:11.164316 18238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.115.195:37525
I20260811 18:54:11.166594 28111 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:54:11.172789 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.168634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.27.115.254:36639)
0811 18:54:11.168819 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.168825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.168992 (+ 167us) server_negotiation.cc:407] Connection header received
0811 18:54:11.169152 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.169155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.169203 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.169293 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:11.169582 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.170118 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.171080 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.171321 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.171434 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.172303 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:11.172309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:11.172316 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:11.172329 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:11.172348 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:11.172355 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:11.172436 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.172563 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.172621 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:54:11.176007 18238 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:54:11.179137 18238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:54:11.194471 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.188604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42039 (local address 127.0.0.1:33054)
0811 18:54:11.189117 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.189166 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.189306 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.190152 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.190159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.190187 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:11.190350 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.190358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.191521 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.191523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.192445 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.192456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.193786 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.193789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.193910 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.193952 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.194168 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.194310 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":89,"threads_started":1}
I20260811 18:54:11.194561 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.188745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.27.115.194:42039)
0811 18:54:11.189177 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.189228 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.189261 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:54:11.189523 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.189532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.189767 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.189905 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:11.190487 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.191397 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.192576 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.193398 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.193520 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.193716 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.194234 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.194373 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":138,"threads_started":1}
I20260811 18:54:11.194993 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.188670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:41157 (local address 127.0.0.1:34392)
0811 18:54:11.189412 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.189429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.189522 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.190591 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.190595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.190614 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:11.190932 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.190941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.192404 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.192408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.193317 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.193326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.194680 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.194683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.194785 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.194804 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.194848 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.194897 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":188,"thread_start_us":152,"threads_started":1}
I20260811 18:54:11.195864 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.189671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.27.115.195:41157)
0811 18:54:11.190089 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.190103 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.190150 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:54:11.190240 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.190246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.190419 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.190691 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:11.191060 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.192267 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.193458 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.194532 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.195344 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.195535 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.195586 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.195697 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":180,"threads_started":1}
I20260811 18:54:11.195924 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.189367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43489 (local address 127.0.0.1:41220)
0811 18:54:11.189732 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.189745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.189833 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.190607 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.190609 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.190621 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:11.190741 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.190747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.192484 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.192490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.193658 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.193669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.195485 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.195489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.195600 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.195622 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.195706 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.195792 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260811 18:54:11.197523 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.189720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.27.115.193:43489)
0811 18:54:11.190226 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.190232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.190252 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:54:11.190318 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.190322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.190465 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.191134 (+ 669us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:11.191278 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.192372 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.193804 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.195331 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.195462 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.195631 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.197096 (+ 1465us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.197209 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":118,"threads_started":1}
I20260811 18:54:11.198167 18528 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc (DEFAULT_TABLE table=test-table [id=49b714478c224ec9850c8aec8ad94458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:11.198650 18670 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc (DEFAULT_TABLE table=test-table [id=49b714478c224ec9850c8aec8ad94458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:11.199502 18528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d9eb76b1f41969cfe6448bf4396dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:11.199725 18670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d9eb76b1f41969cfe6448bf4396dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:11.200387 18381 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc (DEFAULT_TABLE table=test-table [id=49b714478c224ec9850c8aec8ad94458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:54:11.201445 18381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3d9eb76b1f41969cfe6448bf4396dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:11.207858 18759 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Bootstrap starting.
I20260811 18:54:11.209201 18761 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Bootstrap starting.
I20260811 18:54:11.210350 18759 tablet_bootstrap.cc:654] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:11.210598 18760 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Bootstrap starting.
I20260811 18:54:11.211299 18759 log.cc:824] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:11.211496 18761 tablet_bootstrap.cc:654] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:11.212368 18761 log.cc:824] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:11.212700 18760 tablet_bootstrap.cc:654] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:11.213531 18760 log.cc:824] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:11.214491 18759 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: No bootstrap required, opened a new log
I20260811 18:54:11.214715 18761 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: No bootstrap required, opened a new log
I20260811 18:54:11.214728 18759 ts_tablet_manager.cc:1397] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 18:54:11.214920 18761 ts_tablet_manager.cc:1397] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 18:54:11.215771 18760 tablet_bootstrap.cc:492] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: No bootstrap required, opened a new log
I20260811 18:54:11.215996 18760 ts_tablet_manager.cc:1397] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:54:11.218997 18759 raft_consensus.cc:348] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.219421 18759 raft_consensus.cc:374] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:11.219492 18759 raft_consensus.cc:729] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0ac0e2767d4bd0aef8faa84330a9a3, State: Initialized, Role: FOLLOWER
I20260811 18:54:11.219586 18761 raft_consensus.cc:348] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.219866 18761 raft_consensus.cc:374] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:11.219921 18761 raft_consensus.cc:729] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f5ee785c854dceac306ee68f929f2e, State: Initialized, Role: FOLLOWER
I20260811 18:54:11.220041 18759 consensus_queue.cc:260] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.220355 18761 consensus_queue.cc:260] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.220666 18760 raft_consensus.cc:348] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.220934 18760 raft_consensus.cc:374] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:11.220994 18760 raft_consensus.cc:729] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c91e6133314e3dbbbadf1f69aa61a2, State: Initialized, Role: FOLLOWER
I20260811 18:54:11.221484 18761 ts_tablet_manager.cc:1428] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:54:11.221444 18760 consensus_queue.cc:260] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.221518 18759 ts_tablet_manager.cc:1428] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 18:54:11.221941 18457 heartbeater.cc:503] Master 127.27.115.254:36639 was elected leader, sending a full tablet report...
I20260811 18:54:11.221941 18602 heartbeater.cc:503] Master 127.27.115.254:36639 was elected leader, sending a full tablet report...
I20260811 18:54:11.222517 18760 ts_tablet_manager.cc:1428] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 18:54:11.223145 18740 heartbeater.cc:503] Master 127.27.115.254:36639 was elected leader, sending a full tablet report...
W20260811 18:54:11.239465 18458 tablet.cc:2369] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:11.333173 18766 raft_consensus.cc:482] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:54:11.333474 18766 raft_consensus.cc:504] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.334772 18766 leader_election.cc:288] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b0ac0e2767d4bd0aef8faa84330a9a3 (127.27.115.194:42039), f8c91e6133314e3dbbbadf1f69aa61a2 (127.27.115.195:41157)
I20260811 18:54:11.339010 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.335323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:54073 (local address 127.27.115.194:42039)
0811 18:54:11.335463 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.335467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.335674 (+ 207us) server_negotiation.cc:407] Connection header received
0811 18:54:11.335849 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.335852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.335897 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.335989 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:11.336304 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.336823 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.337966 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.338382 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.338523 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.338598 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.338795 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.338872 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:54:11.339030 18768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.335146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.194:42039 (local address 127.27.115.193:54073)
0811 18:54:11.335575 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.335587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.335718 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.336003 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.336006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.336024 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:11.336179 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.336184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.336948 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.336954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.337828 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.337838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.338522 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.338526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.338641 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.338665 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.338762 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.338848 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":1,"thread_start_us":131,"threads_started":1}
I20260811 18:54:11.339041 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.335250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.115.193:44839 (local address 127.27.115.195:41157)
0811 18:54:11.335405 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.335412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.335603 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:54:11.335786 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.335793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.335836 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.335924 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:54:11.336258 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.336774 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.337962 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.338457 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.338590 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.338640 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.338834 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.338904 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:54:11.339061 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.335066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.195:41157 (local address 127.27.115.193:44839)
0811 18:54:11.335500 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.335512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:11.335651 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:11.335940 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:11.335942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:11.335959 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:54:11.336135 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.336141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.336896 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.336899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.337825 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:11.337835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.338588 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.338592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:11.338690 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.338721 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:54:11.338817 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:54:11.338894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":75,"thread_start_us":117,"threads_started":1}
I20260811 18:54:11.339736 18551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d9eb76b1f41969cfe6448bf4396dc" candidate_uuid: "07f5ee785c854dceac306ee68f929f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" is_pre_election: true
I20260811 18:54:11.339836 18692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d9eb76b1f41969cfe6448bf4396dc" candidate_uuid: "07f5ee785c854dceac306ee68f929f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" is_pre_election: true
I20260811 18:54:11.340127 18551 raft_consensus.cc:2454] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5ee785c854dceac306ee68f929f2e in term 0.
I20260811 18:54:11.340160 18692 raft_consensus.cc:2454] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f5ee785c854dceac306ee68f929f2e in term 0.
I20260811 18:54:11.340706 18329 leader_election.cc:302] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07f5ee785c854dceac306ee68f929f2e, 0b0ac0e2767d4bd0aef8faa84330a9a3, f8c91e6133314e3dbbbadf1f69aa61a2; no voters:
I20260811 18:54:11.341027 18766 raft_consensus.cc:2781] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:54:11.341107 18766 raft_consensus.cc:482] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:54:11.341159 18766 raft_consensus.cc:3037] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:11.342476 18766 raft_consensus.cc:504] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.342948 18766 leader_election.cc:288] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [CANDIDATE]: Term 1 election: Requested vote from peers 0b0ac0e2767d4bd0aef8faa84330a9a3 (127.27.115.194:42039), f8c91e6133314e3dbbbadf1f69aa61a2 (127.27.115.195:41157)
I20260811 18:54:11.343380 18551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d9eb76b1f41969cfe6448bf4396dc" candidate_uuid: "07f5ee785c854dceac306ee68f929f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3"
I20260811 18:54:11.343410 18692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3d9eb76b1f41969cfe6448bf4396dc" candidate_uuid: "07f5ee785c854dceac306ee68f929f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2"
I20260811 18:54:11.343564 18692 raft_consensus.cc:3037] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:11.343575 18551 raft_consensus.cc:3037] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:11.344762 18692 raft_consensus.cc:2454] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5ee785c854dceac306ee68f929f2e in term 1.
I20260811 18:54:11.344784 18551 raft_consensus.cc:2454] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f5ee785c854dceac306ee68f929f2e in term 1.
I20260811 18:54:11.345252 18327 leader_election.cc:302] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f5ee785c854dceac306ee68f929f2e, f8c91e6133314e3dbbbadf1f69aa61a2; no voters:
I20260811 18:54:11.345461 18766 raft_consensus.cc:2781] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:11.345739 18766 raft_consensus.cc:686] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [term 1 LEADER]: Becoming Leader. State: Replica: 07f5ee785c854dceac306ee68f929f2e, State: Running, Role: LEADER
I20260811 18:54:11.346199 18766 consensus_queue.cc:237] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } }
I20260811 18:54:11.349841 18236 catalog_manager.cc:5495] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f5ee785c854dceac306ee68f929f2e (127.27.115.193). New cstate: current_term: 1 leader_uuid: "07f5ee785c854dceac306ee68f929f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f5ee785c854dceac306ee68f929f2e" member_type: VOTER last_known_addr { host: "127.27.115.193" port: 43489 } health_report { overall_health: HEALTHY } } }
I20260811 18:54:11.389187 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.385590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.27.115.193:43489)
0811 18:54:11.385729 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:11.385734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:11.385749 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:11.385891 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:11.385893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:11.385955 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:11.386069 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:11.386348 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.386852 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.387710 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:11.387925 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:11.388053 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:11.388358 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:11.388780 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:11.388915 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:11.388979 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:54:11.389035 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:54:11.395246 18692 raft_consensus.cc:1261] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 1 FOLLOWER]: Refusing update from remote peer 07f5ee785c854dceac306ee68f929f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:11.395246 18551 raft_consensus.cc:1261] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 1 FOLLOWER]: Refusing update from remote peer 07f5ee785c854dceac306ee68f929f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:11.396142 18770 consensus_queue.cc:1035] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8c91e6133314e3dbbbadf1f69aa61a2" member_type: VOTER last_known_addr { host: "127.27.115.195" port: 41157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:11.396420 18766 consensus_queue.cc:1035] T 8f3d9eb76b1f41969cfe6448bf4396dc P 07f5ee785c854dceac306ee68f929f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0ac0e2767d4bd0aef8faa84330a9a3" member_type: VOTER last_known_addr { host: "127.27.115.194" port: 42039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:54:11.399207 18741 tablet.cc:2369] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:11.405726 18773 mvcc.cc:205] Tried to move back new op lower bound from 7317399352906960896 to 7317399352716980224. Current Snapshot: MvccSnapshot[applied={T|T < 7317399352906960896 or (T in {7317399352906960896})}]
I20260811 18:54:11.412667 18236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:11.412884 18236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:54:11.415326 18236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2dc2a07a3114ee485b754cd46a058a6: Sending DeleteTablet for 3 replicas of tablet 8f3d9eb76b1f41969cfe6448bf4396dc
I20260811 18:54:11.416193 18528 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:11 UTC) from {username='slave'} at 127.0.0.1:33054
I20260811 18:54:11.416322 18381 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:11 UTC) from {username='slave'} at 127.0.0.1:41220
I20260811 18:54:11.416740 18784 tablet_replica.cc:330] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: stopping tablet replica
I20260811 18:54:11.416745 18670 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3d9eb76b1f41969cfe6448bf4396dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:54:11 UTC) from {username='slave'} at 127.0.0.1:34392
I20260811 18:54:11.417114 18784 raft_consensus.cc:2229] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:11.417323 18786 tablet_replica.cc:330] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: stopping tablet replica
I20260811 18:54:11.417479 18784 raft_consensus.cc:2258] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:11.417915 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18307
I20260811 18:54:11.419498 18784 ts_tablet_manager.cc:1905] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:11.417950 18786 raft_consensus.cc:2229] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:54:11.420819 18786 raft_consensus.cc:2258] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:54:11.422608 18784 ts_tablet_manager.cc:1918] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:11.422720 18784 log.cc:1196] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-1/wal/wals/8f3d9eb76b1f41969cfe6448bf4396dc
I20260811 18:54:11.423035 18784 ts_tablet_manager.cc:1939] T 8f3d9eb76b1f41969cfe6448bf4396dc P 0b0ac0e2767d4bd0aef8faa84330a9a3: Deleting consensus metadata
I20260811 18:54:11.423756 18786 ts_tablet_manager.cc:1905] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:54:11.428027 18224 catalog_manager.cc:4841] TS 0b0ac0e2767d4bd0aef8faa84330a9a3 (127.27.115.194:42039): tablet 8f3d9eb76b1f41969cfe6448bf4396dc (table test-table [id=49b714478c224ec9850c8aec8ad94458]) successfully deleted
I20260811 18:54:11.428110 18786 ts_tablet_manager.cc:1918] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:54:11.428395 18786 log.cc:1196] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Deleting WAL directory at /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474173680671-28111-0/minicluster-data/ts-2/wal/wals/8f3d9eb76b1f41969cfe6448bf4396dc
I20260811 18:54:11.428822 18786 ts_tablet_manager.cc:1939] T 8f3d9eb76b1f41969cfe6448bf4396dc P f8c91e6133314e3dbbbadf1f69aa61a2: Deleting consensus metadata
I20260811 18:54:11.430286 18223 catalog_manager.cc:4841] TS f8c91e6133314e3dbbbadf1f69aa61a2 (127.27.115.195:41157): tablet 8f3d9eb76b1f41969cfe6448bf4396dc (table test-table [id=49b714478c224ec9850c8aec8ad94458]) successfully deleted
W20260811 18:54:11.434885 18224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.115.193:43489 (error 108)
I20260811 18:54:11.435897 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18461
I20260811 18:54:11.436174 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:11.435830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.193:43489 (local address 127.0.0.1:41230)
0811 18:54:11.435980 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:11.436069 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.193:43489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 18:54:11.439221 18224 catalog_manager.cc:4584] TS 07f5ee785c854dceac306ee68f929f2e (127.27.115.193:43489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f3d9eb76b1f41969cfe6448bf4396dc: Network error: Client connection negotiation failed: client connection to 127.27.115.193:43489: connect: Connection refused (error 111)
I20260811 18:54:11.444918 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18605
I20260811 18:54:11.455229 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18203
2026-08-11T18:54:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1262 ms)
[----------] 6 tests from EncryptionPolicyTest (11738 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:11 dist-test-slave-9jjr krb5kdc[18792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:11 dist-test-slave-9jjr krb5kdc[18792](info): set up 2 sockets
Aug 11 18:54:11 dist-test-slave-9jjr krb5kdc[18792](info): commencing operation
krb5kdc: starting...
W20260811 18:54:12.591816 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:12 dist-test-slave-9jjr krb5kdc[18792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:12.734529 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:12.854031 18805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:12.854501 18805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:12.854614 18805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:12.858976 18805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:12.859115 18805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:12.859153 18805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:12.859195 18805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:12.859256 18805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:12.868211 18805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:12.870048 18805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:12.871610 18805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:12.881652 18813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:12.881695 18816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:12.881655 18811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:12.882102 18805 server_base.cc:1034] running on GCE node
I20260811 18:54:12.882827 18805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:12.884137 18805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:12.885361 18805 hybrid_clock.cc:648] HybridClock initialized: now 1786474452885343 us; error 39 us; skew 500 ppm
Aug 11 18:54:12 dist-test-slave-9jjr krb5kdc[18792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474452, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:12.888579 18805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:12.889679 18805 webserver.cc:460] Webserver started at http://127.0.0.1:40063/ using document root <none> and password file <none>
I20260811 18:54:12.890360 18805 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:12.890460 18805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:12.890718 18805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:12.892920 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "808845bcf247429c8c29cf9704ae14aa"
format_stamp: "Formatted at 2026-08-11 18:54:12 on dist-test-slave-9jjr"
server_key: "2e04acd1ecdd00c7ecd02ef6ceaa62d1"
server_key_iv: "411131aada86bde39fb482911304bbba"
server_key_version: "encryptionkey@0"
I20260811 18:54:12.893523 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "808845bcf247429c8c29cf9704ae14aa"
format_stamp: "Formatted at 2026-08-11 18:54:12 on dist-test-slave-9jjr"
server_key: "2e04acd1ecdd00c7ecd02ef6ceaa62d1"
server_key_iv: "411131aada86bde39fb482911304bbba"
server_key_version: "encryptionkey@0"
I20260811 18:54:12.897759 18805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:54:12.900789 18824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:12.902515 18805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:12.902691 18805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "808845bcf247429c8c29cf9704ae14aa"
format_stamp: "Formatted at 2026-08-11 18:54:12 on dist-test-slave-9jjr"
server_key: "2e04acd1ecdd00c7ecd02ef6ceaa62d1"
server_key_iv: "411131aada86bde39fb482911304bbba"
server_key_version: "encryptionkey@0"
I20260811 18:54:12.902840 18805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:12.923915 18805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:12.927695 18805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:12.927958 18805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:12.936075 18805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38419
I20260811 18:54:12.936101 18879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38419 every 8 connection(s)
I20260811 18:54:12.937326 18805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:12.940361 18880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:12.942924 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18805
I20260811 18:54:12.943049 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:12.943413 28111 external_mini_cluster.cc:1428] Setting key 042e86fbc6f72aedc6fa04dce48048fb
I20260811 18:54:12.946938 18880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa: Bootstrap starting.
I20260811 18:54:12.949620 18880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:12.950556 18880 log.cc:824] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:54:12 dist-test-slave-9jjr krb5kdc[18792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:54:12.953109 18880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa: No bootstrap required, opened a new log
I20260811 18:54:12.958045 18880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } }
I20260811 18:54:12.958454 18880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:12.958523 18880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808845bcf247429c8c29cf9704ae14aa, State: Initialized, Role: FOLLOWER
I20260811 18:54:12.959071 18880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } }
I20260811 18:54:12.959288 18880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:12.959370 18880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:12.959481 18880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:12.960858 18883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:12.945006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59671 (local address 127.0.0.1:38419)
0811 18:54:12.945473 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:12.945494 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:12.945531 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:54:12.946250 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:12.946268 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:12.946616 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:12.946887 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:12.947870 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:12.948938 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:12.949957 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:12.950536 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:12.950732 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:12.953825 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:12.953831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:12.953851 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:12.953891 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:12.956928 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:12.957654 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:12.957661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:12.957668 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:12.957855 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:12.958268 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:12.958272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:12.958273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:12.958877 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:12.959105 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:12.959436 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 18:54:12.959710 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":171,"threads_started":1}
I20260811 18:54:12.960829 18880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } }
I20260811 18:54:12.961259 18880 leader_election.cc:302] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 808845bcf247429c8c29cf9704ae14aa; no voters:
I20260811 18:54:12.961644 18880 leader_election.cc:288] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:12.961805 18885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:12.962060 18885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [term 1 LEADER]: Becoming Leader. State: Replica: 808845bcf247429c8c29cf9704ae14aa, State: Running, Role: LEADER
I20260811 18:54:12.962476 18885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } }
I20260811 18:54:12.963076 18880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:12.964339 18887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "808845bcf247429c8c29cf9704ae14aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } } }
I20260811 18:54:12.964474 18887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:12.964502 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:12.964841 18886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 808845bcf247429c8c29cf9704ae14aa. Latest consensus state: current_term: 1 leader_uuid: "808845bcf247429c8c29cf9704ae14aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "808845bcf247429c8c29cf9704ae14aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38419 } } }
I20260811 18:54:12.964936 18886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:12.965044 18896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:12.968408 18896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:12.975761 18883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:12.967023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34915 (local address 127.0.0.1:38419)
0811 18:54:12.967140 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:12.967144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:12.967222 (+ 78us) server_negotiation.cc:407] Connection header received
0811 18:54:12.967400 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:12.967403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:12.967452 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:12.967538 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:12.968239 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:12.968862 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:12.969696 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:12.969931 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:12.970047 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:12.971896 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:12.971901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:12.971903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:12.971931 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:12.973714 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:12.974248 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:12.974252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:12.974254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:12.974349 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:12.974738 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:12.974742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:12.974743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:12.974980 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:12.975137 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:12.975477 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 18:54:12.975588 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:54:12.976023 18896 catalog_manager.cc:1347] Generated new cluster ID: 8d25ed1565484807af3cd952c7b30cef
I20260811 18:54:12.976085 18896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:12.991240 18896 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:12.992254 18896 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:12.999557 18896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 808845bcf247429c8c29cf9704ae14aa: Generated new TSK 0
I20260811 18:54:13.000422 18896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:13.009140 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18805
W20260811 18:54:13.019485 18789 connection.cc:535] client connection to 127.0.0.1:38419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38419): Connection reset by peer (error 104)
2026-08-11T18:54:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:13 dist-test-slave-9jjr krb5kdc[18914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:13 dist-test-slave-9jjr krb5kdc[18914](info): set up 2 sockets
Aug 11 18:54:13 dist-test-slave-9jjr krb5kdc[18914](info): commencing operation
krb5kdc: starting...
W20260811 18:54:14.125519 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:14 dist-test-slave-9jjr krb5kdc[18914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:14.265873 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:14.387768 18928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:14.388084 18928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:14.388130 18928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:14.392215 18928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:14.392293 18928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:14.392313 18928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:14.392330 18928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:14.392346 18928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:14.397984 18928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:14.399207 18928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:14.400246 18928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:14.410032 18936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:14.410190 18933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:14.410190 18934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:14.410570 18928 server_base.cc:1034] running on GCE node
I20260811 18:54:14.411227 18928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:14.412413 18928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:14.413583 18928 hybrid_clock.cc:648] HybridClock initialized: now 1786474454413552 us; error 46 us; skew 500 ppm
Aug 11 18:54:14 dist-test-slave-9jjr krb5kdc[18914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:14.416671 18928 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:14.417747 18928 webserver.cc:460] Webserver started at http://127.0.0.1:40809/ using document root <none> and password file <none>
I20260811 18:54:14.418453 18928 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:14.418552 18928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:14.418817 18928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:14.421133 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "8665e8da38e04e95a3fc2791fa9804e0"
format_stamp: "Formatted at 2026-08-11 18:54:14 on dist-test-slave-9jjr"
server_key: "954b20a0202084af963f07053705f989"
server_key_iv: "97e8f06fdbe357f57d140bab8ea3669f"
server_key_version: "encryptionkey@0"
I20260811 18:54:14.421782 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "8665e8da38e04e95a3fc2791fa9804e0"
format_stamp: "Formatted at 2026-08-11 18:54:14 on dist-test-slave-9jjr"
server_key: "954b20a0202084af963f07053705f989"
server_key_iv: "97e8f06fdbe357f57d140bab8ea3669f"
server_key_version: "encryptionkey@0"
I20260811 18:54:14.426319 18928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:54:14.429266 18944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:14.431023 18928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:14.431242 18928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "8665e8da38e04e95a3fc2791fa9804e0"
format_stamp: "Formatted at 2026-08-11 18:54:14 on dist-test-slave-9jjr"
server_key: "954b20a0202084af963f07053705f989"
server_key_iv: "97e8f06fdbe357f57d140bab8ea3669f"
server_key_version: "encryptionkey@0"
I20260811 18:54:14.431393 18928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:14.452219 18928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:14.456247 18928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:14.456540 18928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:14.465458 18928 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35017
I20260811 18:54:14.465487 19000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35017 every 8 connection(s)
I20260811 18:54:14.466562 18928 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:14.469640 19001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:14.474643 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 18928
I20260811 18:54:14.474821 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:14.475129 28111 external_mini_cluster.cc:1428] Setting key bf610a8a0a0aae85bc152d2f1d2fd3a3
I20260811 18:54:14.476912 19001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: Bootstrap starting.
I20260811 18:54:14.479604 19001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:14.480481 19001 log.cc:824] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:14.482671 19001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: No bootstrap required, opened a new log
I20260811 18:54:14.486104 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:14.476887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56251 (local address 127.0.0.1:35017)
0811 18:54:14.477452 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:14.477471 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:14.477514 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:54:14.478495 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:14.478518 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:14.478945 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:14.479349 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:14.479889 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:14.481076 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:14.482212 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:14.482745 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:14.482940 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:14.484021 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:14.484027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:14.484042 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:14.484528 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:14.484594 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:14.484620 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:14.484744 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:14.485104 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 18:54:14.485293 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":166,"threads_started":1}
I20260811 18:54:14.487236 19001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260811 18:54:14.487571 19001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:14.487634 19001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8665e8da38e04e95a3fc2791fa9804e0, State: Initialized, Role: FOLLOWER
I20260811 18:54:14.488085 19001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260811 18:54:14.488266 19001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:14.488366 19001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:14.488477 19001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:14.489763 19001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260811 18:54:14.489820 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:14.490196 19001 leader_election.cc:302] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8665e8da38e04e95a3fc2791fa9804e0; no voters:
I20260811 18:54:14.490581 19001 leader_election.cc:288] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:14.490725 19006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:14.490986 19006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [term 1 LEADER]: Becoming Leader. State: Replica: 8665e8da38e04e95a3fc2791fa9804e0, State: Running, Role: LEADER
I20260811 18:54:14.491472 19006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260811 18:54:14.492225 19001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:14.493542 19008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8665e8da38e04e95a3fc2791fa9804e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } } }
I20260811 18:54:14.493692 19008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:14.494225 19007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8665e8da38e04e95a3fc2791fa9804e0. Latest consensus state: current_term: 1 leader_uuid: "8665e8da38e04e95a3fc2791fa9804e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8665e8da38e04e95a3fc2791fa9804e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } } }
I20260811 18:54:14.494391 19007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:14.498097 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:14.492459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36917 (local address 127.0.0.1:35017)
0811 18:54:14.492583 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:14.492588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:14.493332 (+ 744us) server_negotiation.cc:407] Connection header received
0811 18:54:14.493570 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:14.493573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:14.493629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:14.493686 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:14.494257 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:14.495052 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:14.496191 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:14.496469 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:14.496599 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:14.497547 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:14.497551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:14.497554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:14.497576 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:14.497601 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:14.497612 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:14.497733 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:14.497893 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:54:14.497932 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 18:54:14.498173 19029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:14.498250 19029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:14.498334 19023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:14.499295 19023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:14.505803 19023 catalog_manager.cc:1347] Generated new cluster ID: 3246dd56cf774374823796d087aa9381
I20260811 18:54:14.505916 19023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:14.515520 19023 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:14.516930 19023 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:14.522019 19023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8665e8da38e04e95a3fc2791fa9804e0: Generated new TSK 0
I20260811 18:54:14.523007 19023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:14.530009 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 18928
W20260811 18:54:14.540041 18912 connection.cc:535] client connection to 127.0.0.1:35017 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35017): Connection reset by peer (error 104)
2026-08-11T18:54:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:14 dist-test-slave-9jjr krb5kdc[19035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:14 dist-test-slave-9jjr krb5kdc[19035](info): set up 2 sockets
Aug 11 18:54:14 dist-test-slave-9jjr krb5kdc[19035](info): commencing operation
krb5kdc: starting...
W20260811 18:54:15.665246 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:15 dist-test-slave-9jjr krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:15.811681 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:15.928718 19050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:15.929055 19050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:15.929147 19050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:15.933223 19050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:15.933343 19050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:15.933384 19050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:15.933421 19050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:15.933457 19050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:15.939556 19050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:15.940757 19050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:15.941838 19050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:15.951467 19057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:15.951467 19055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:15.951550 19050 server_base.cc:1034] running on GCE node
W20260811 18:54:15.951467 19059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:15.952360 19050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:15.953552 19050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:15.954732 19050 hybrid_clock.cc:648] HybridClock initialized: now 1786474455954700 us; error 45 us; skew 500 ppm
Aug 11 18:54:15 dist-test-slave-9jjr krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474455, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:15.963160 19050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:15.964216 19050 webserver.cc:460] Webserver started at http://127.0.0.1:37299/ using document root <none> and password file <none>
I20260811 18:54:15.964826 19050 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:15.964895 19050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:15.965165 19050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:15.967409 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "f376380e2d7b4cbba0ec723fdb3076cd"
format_stamp: "Formatted at 2026-08-11 18:54:15 on dist-test-slave-9jjr"
server_key: "0ff3b09b56b25a826202f80bc56c34e6"
server_key_iv: "6c878d335be8f945f0bb4b87a372833f"
server_key_version: "encryptionkey@0"
I20260811 18:54:15.968022 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "f376380e2d7b4cbba0ec723fdb3076cd"
format_stamp: "Formatted at 2026-08-11 18:54:15 on dist-test-slave-9jjr"
server_key: "0ff3b09b56b25a826202f80bc56c34e6"
server_key_iv: "6c878d335be8f945f0bb4b87a372833f"
server_key_version: "encryptionkey@0"
I20260811 18:54:15.972119 19050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:15.974792 19066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:15.976151 19050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:15.976321 19050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "f376380e2d7b4cbba0ec723fdb3076cd"
format_stamp: "Formatted at 2026-08-11 18:54:15 on dist-test-slave-9jjr"
server_key: "0ff3b09b56b25a826202f80bc56c34e6"
server_key_iv: "6c878d335be8f945f0bb4b87a372833f"
server_key_version: "encryptionkey@0"
I20260811 18:54:15.976459 19050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:15.981482 19050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:15.984910 19050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:15.985157 19050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:15.993018 19050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38079
I20260811 18:54:15.993044 19121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38079 every 8 connection(s)
I20260811 18:54:15.994024 19050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:15.997150 19123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:15.999657 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19050
I20260811 18:54:15.999814 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:16.000102 28111 external_mini_cluster.cc:1428] Setting key 25d99ab17c9870a84828d221ef461ecc
I20260811 18:54:16.003800 19123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd: Bootstrap starting.
I20260811 18:54:16.005937 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:16.001681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35675 (local address 127.0.0.1:38079)
0811 18:54:16.002292 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:16.002310 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:16.002378 (+ 68us) server_negotiation.cc:407] Connection header received
0811 18:54:16.003096 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:16.003114 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:16.003432 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:16.003799 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:16.004657 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:16.004661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:16.004673 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:16.004685 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:16.004740 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:16.004760 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:16.004921 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:16.004996 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:54:16.005166 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":176,"threads_started":1}
I20260811 18:54:16.006623 19123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:16.007507 19123 log.cc:824] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:16.009061 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:16.009866 19123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd: No bootstrap required, opened a new log
I20260811 18:54:16.013085 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:16.011321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59317 (local address 127.0.0.1:38079)
0811 18:54:16.011473 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:16.011479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:16.011658 (+ 179us) server_negotiation.cc:407] Connection header received
0811 18:54:16.011806 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:16.011808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:16.011839 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:16.011907 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:16.012670 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:16.012673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:16.012675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:16.012687 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:16.012713 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:16.012724 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:16.012812 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:16.012899 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 18:54:16.012934 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:54:16.014834 19123 raft_consensus.cc:348] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } }
I20260811 18:54:16.015132 19123 raft_consensus.cc:374] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:16.015194 19123 raft_consensus.cc:729] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f376380e2d7b4cbba0ec723fdb3076cd, State: Initialized, Role: FOLLOWER
I20260811 18:54:16.015688 19123 consensus_queue.cc:260] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } }
I20260811 18:54:16.015846 19123 raft_consensus.cc:388] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:16.015933 19123 raft_consensus.cc:482] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:16.016038 19123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:16.017109 19123 raft_consensus.cc:504] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } }
I20260811 18:54:16.017490 19123 leader_election.cc:302] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f376380e2d7b4cbba0ec723fdb3076cd; no voters:
I20260811 18:54:16.017848 19123 leader_election.cc:288] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:16.017958 19135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:16.018263 19135 raft_consensus.cc:686] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [term 1 LEADER]: Becoming Leader. State: Replica: f376380e2d7b4cbba0ec723fdb3076cd, State: Running, Role: LEADER
I20260811 18:54:16.018695 19135 consensus_queue.cc:237] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } }
I20260811 18:54:16.019312 19123 sys_catalog.cc:564] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:16.020546 19137 sys_catalog.cc:455] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f376380e2d7b4cbba0ec723fdb3076cd. Latest consensus state: current_term: 1 leader_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } } }
I20260811 18:54:16.020546 19136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f376380e2d7b4cbba0ec723fdb3076cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38079 } } }
I20260811 18:54:16.020704 19137 sys_catalog.cc:458] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:16.020704 19136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:16.021159 19144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:16.024590 19144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:16.032006 19144 catalog_manager.cc:1347] Generated new cluster ID: 022f618c14f04c0ea386b5f43b69cef7
I20260811 18:54:16.032119 19144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:16.041332 19144 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:16.041720 19144 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:16.047887 19144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f376380e2d7b4cbba0ec723fdb3076cd: Generated new TSK 0
I20260811 18:54:16.048739 19144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:16.059968 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19050
2026-08-11T18:54:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:16 dist-test-slave-9jjr krb5kdc[19157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:16 dist-test-slave-9jjr krb5kdc[19157](info): set up 2 sockets
Aug 11 18:54:16 dist-test-slave-9jjr krb5kdc[19157](info): commencing operation
krb5kdc: starting...
W20260811 18:54:17.185940 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:17.326462 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:17.446487 19173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:17.446806 19173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:17.446851 19173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:17.450927 19173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:17.450997 19173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:17.451016 19173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:17.451033 19173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:17.451049 19173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:17.456545 19173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:17.457728 19173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:17.458855 19173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:17.468305 19181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:17.468294 19179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:17.468299 19178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:17.468626 19173 server_base.cc:1034] running on GCE node
I20260811 18:54:17.469322 19173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:17.470624 19173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:17.471838 19173 hybrid_clock.cc:648] HybridClock initialized: now 1786474457471807 us; error 54 us; skew 500 ppm
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:17.475028 19173 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:17.476099 19173 webserver.cc:460] Webserver started at http://127.0.0.1:35213/ using document root <none> and password file <none>
I20260811 18:54:17.476658 19173 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:17.476720 19173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:17.476905 19173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:17.479101 19173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "d3c6404fd4204b239cf5509198d9643a"
format_stamp: "Formatted at 2026-08-11 18:54:17 on dist-test-slave-9jjr"
server_key: "0c1cf87c38560082cf7db3238ff41943"
server_key_iv: "de7430a06ab080263bd2fa9f1f8b352d"
server_key_version: "encryptionkey@0"
I20260811 18:54:17.479648 19173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "d3c6404fd4204b239cf5509198d9643a"
format_stamp: "Formatted at 2026-08-11 18:54:17 on dist-test-slave-9jjr"
server_key: "0c1cf87c38560082cf7db3238ff41943"
server_key_iv: "de7430a06ab080263bd2fa9f1f8b352d"
server_key_version: "encryptionkey@0"
I20260811 18:54:17.483958 19173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:17.487155 19188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:17.488711 19173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:17.488869 19173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "d3c6404fd4204b239cf5509198d9643a"
format_stamp: "Formatted at 2026-08-11 18:54:17 on dist-test-slave-9jjr"
server_key: "0c1cf87c38560082cf7db3238ff41943"
server_key_iv: "de7430a06ab080263bd2fa9f1f8b352d"
server_key_version: "encryptionkey@0"
I20260811 18:54:17.489056 19173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:17.506567 19173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:17.510479 19173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:17.510733 19173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:17.519038 19173 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41965
I20260811 18:54:17.519068 19244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41965 every 8 connection(s)
I20260811 18:54:17.520190 19173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:17.523346 19246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:17.525323 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19173
I20260811 18:54:17.525486 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:17.525833 28111 external_mini_cluster.cc:1428] Setting key 2636d256127c2aa8e5579909a5de3369
I20260811 18:54:17.530717 19246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a: Bootstrap starting.
I20260811 18:54:17.533242 19246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:17.534310 19246 log.cc:824] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:54:17.536995 19246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a: No bootstrap required, opened a new log
I20260811 18:54:17.541991 19246 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260811 18:54:17.542323 19246 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:17.542399 19246 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c6404fd4204b239cf5509198d9643a, State: Initialized, Role: FOLLOWER
I20260811 18:54:17.542526 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:17.527549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45165 (local address 127.0.0.1:41965)
0811 18:54:17.528081 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:17.528096 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:17.528134 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:54:17.528800 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:17.528818 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:17.529170 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:17.529520 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:17.530466 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:17.531558 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:17.532450 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:17.532975 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:17.533167 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:17.536517 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:17.536523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:17.536540 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:17.536571 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:17.538792 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:17.539504 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:17.539510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:17.539516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:17.539655 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:17.540016 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:17.540019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:17.540020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:17.540513 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:17.540739 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:17.541058 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 18:54:17.541456 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":162,"threads_started":1}
I20260811 18:54:17.542989 19246 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260811 18:54:17.543154 19246 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:17.543249 19246 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:17.543376 19246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:17.544746 19246 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260811 18:54:17.545138 19246 leader_election.cc:302] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3c6404fd4204b239cf5509198d9643a; no voters:
I20260811 18:54:17.545549 19246 leader_election.cc:288] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:17.545704 19251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:17.545990 19251 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [term 1 LEADER]: Becoming Leader. State: Replica: d3c6404fd4204b239cf5509198d9643a, State: Running, Role: LEADER
I20260811 18:54:17.546236 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:17.546384 19251 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260811 18:54:17.547132 19246 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:17.547636 19251 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c6404fd4204b239cf5509198d9643a. Latest consensus state: current_term: 1 leader_uuid: "d3c6404fd4204b239cf5509198d9643a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } } }
I20260811 18:54:17.547766 19251 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:17.548233 19252 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3c6404fd4204b239cf5509198d9643a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3c6404fd4204b239cf5509198d9643a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } } }
I20260811 18:54:17.548380 19252 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:17.548237 19259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:17.552512 19259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:17.559216 19259 catalog_manager.cc:1347] Generated new cluster ID: 5d47a71ed4434a5db58d2289f912c95e
I20260811 18:54:17.559384 19259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:17.559811 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:17.548785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43981 (local address 127.0.0.1:41965)
0811 18:54:17.548916 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:17.548922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:17.549835 (+ 913us) server_negotiation.cc:407] Connection header received
0811 18:54:17.549966 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:17.549969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:17.550030 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:17.550167 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:17.551131 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:17.551928 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:17.553164 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:17.553519 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:17.553624 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:17.555651 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:17.555657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:17.555660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:17.555693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:17.557572 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:17.558270 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:17.558273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:17.558275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:17.558371 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:17.558713 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:17.558717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:17.558718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:17.558968 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:17.559093 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:17.559521 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 18:54:17.559665 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:54:17.570767 19259 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:17.572075 19259 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:17.578580 19259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3c6404fd4204b239cf5509198d9643a: Generated new TSK 0
I20260811 18:54:17.579432 19259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:17.585471 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19173
2026-08-11T18:54:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19281](info): set up 2 sockets
Aug 11 18:54:17 dist-test-slave-9jjr krb5kdc[19281](info): commencing operation
krb5kdc: starting...
W20260811 18:54:18.711992 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:18 dist-test-slave-9jjr krb5kdc[19281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:18.859753 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:18.984972 19296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:18.985309 19296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:18.985355 19296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:18.989832 19296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:18.989921 19296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:18.989943 19296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:18.989959 19296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:18.989975 19296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:18.995837 19296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:18.997172 19296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:18.998392 19296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:19.007855 19302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:19.007855 19304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:19.007899 19301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:19.008360 19296 server_base.cc:1034] running on GCE node
I20260811 18:54:19.009115 19296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:19.010442 19296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:19.011644 19296 hybrid_clock.cc:648] HybridClock initialized: now 1786474459011623 us; error 49 us; skew 500 ppm
Aug 11 18:54:19 dist-test-slave-9jjr krb5kdc[19281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:19.014640 19296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:19.015587 19296 webserver.cc:460] Webserver started at http://127.0.0.1:33143/ using document root <none> and password file <none>
I20260811 18:54:19.016175 19296 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:19.016242 19296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:19.016422 19296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:19.018688 19296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "5dd890233be9426c8e6e82636d2e673f"
format_stamp: "Formatted at 2026-08-11 18:54:19 on dist-test-slave-9jjr"
server_key: "66658729db3754c8f9ca2325bbb573c3"
server_key_iv: "b2c8b17263514660cd30b5d632d78bab"
server_key_version: "encryptionkey@0"
I20260811 18:54:19.019265 19296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "5dd890233be9426c8e6e82636d2e673f"
format_stamp: "Formatted at 2026-08-11 18:54:19 on dist-test-slave-9jjr"
server_key: "66658729db3754c8f9ca2325bbb573c3"
server_key_iv: "b2c8b17263514660cd30b5d632d78bab"
server_key_version: "encryptionkey@0"
I20260811 18:54:19.023490 19296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 18:54:19.026412 19312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:19.027794 19296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:19.027949 19296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "5dd890233be9426c8e6e82636d2e673f"
format_stamp: "Formatted at 2026-08-11 18:54:19 on dist-test-slave-9jjr"
server_key: "66658729db3754c8f9ca2325bbb573c3"
server_key_iv: "b2c8b17263514660cd30b5d632d78bab"
server_key_version: "encryptionkey@0"
I20260811 18:54:19.028072 19296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:19.049391 19296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:19.052923 19296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:19.053150 19296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:19.060886 19296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40025
I20260811 18:54:19.060906 19367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40025 every 8 connection(s)
I20260811 18:54:19.061964 19296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:19.065287 19368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:19.069053 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19296
I20260811 18:54:19.069227 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:19.069537 28111 external_mini_cluster.cc:1428] Setting key 4c4fad03f11d7ee2d3e0090f919f59e9
I20260811 18:54:19.072237 19368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f: Bootstrap starting.
I20260811 18:54:19.074618 19368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:19.075399 19368 log.cc:824] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:19.077554 19368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f: No bootstrap required, opened a new log
I20260811 18:54:19.079753 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:19.071216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52001 (local address 127.0.0.1:40025)
0811 18:54:19.071735 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:19.071751 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:19.071791 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:54:19.072589 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:19.072609 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:19.072963 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:19.073328 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:19.073760 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:19.074895 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:19.075889 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:19.076425 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:19.076611 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:19.077635 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:19.077640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:19.077675 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:19.078198 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:19.078271 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:19.078296 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:19.078402 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:19.078572 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 18:54:19.078798 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":139,"threads_started":1}
I20260811 18:54:19.080915 19368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 18:54:19.081166 19368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:19.081252 19368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd890233be9426c8e6e82636d2e673f, State: Initialized, Role: FOLLOWER
I20260811 18:54:19.081722 19368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 18:54:19.081879 19368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:19.081969 19368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:19.082072 19368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:19.082780 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:19.083268 19368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 18:54:19.083623 19368 leader_election.cc:302] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dd890233be9426c8e6e82636d2e673f; no voters:
I20260811 18:54:19.083957 19368 leader_election.cc:288] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:19.084113 19373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:19.084364 19373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [term 1 LEADER]: Becoming Leader. State: Replica: 5dd890233be9426c8e6e82636d2e673f, State: Running, Role: LEADER
I20260811 18:54:19.084708 19373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 18:54:19.085487 19368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:19.086699 19376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dd890233be9426c8e6e82636d2e673f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } } }
I20260811 18:54:19.086726 19380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dd890233be9426c8e6e82636d2e673f. Latest consensus state: current_term: 1 leader_uuid: "5dd890233be9426c8e6e82636d2e673f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dd890233be9426c8e6e82636d2e673f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } } }
I20260811 18:54:19.086863 19380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:19.086863 19376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:19.087342 19389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:19.090581 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:19.085252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43281 (local address 127.0.0.1:40025)
0811 18:54:19.085369 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:19.085374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:19.085613 (+ 239us) server_negotiation.cc:407] Connection header received
0811 18:54:19.085813 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:19.085819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:19.085885 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:19.085974 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:19.086640 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:19.087465 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:19.088703 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:19.089039 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:19.089162 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:19.090064 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:19.090070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:19.090075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:19.090091 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:19.090120 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:19.090159 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:19.090238 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:19.090368 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:54:19.090425 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:54:19.090867 19389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:19.096740 19389 catalog_manager.cc:1347] Generated new cluster ID: 387f8f8da3e84dc19352e288ea124dc8
I20260811 18:54:19.096853 19389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:19.115159 19389 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:19.116324 19389 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:19.132179 19389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dd890233be9426c8e6e82636d2e673f: Generated new TSK 0
I20260811 18:54:19.133185 19389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:19.148145 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19296
2026-08-11T18:54:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:19 dist-test-slave-9jjr krb5kdc[19401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:19 dist-test-slave-9jjr krb5kdc[19401](info): set up 2 sockets
Aug 11 18:54:19 dist-test-slave-9jjr krb5kdc[19401](info): commencing operation
krb5kdc: starting...
W20260811 18:54:20.268994 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:20 dist-test-slave-9jjr krb5kdc[19401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:20.419109 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:20.539736 19415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:20.540141 19415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:20.540232 19415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:20.544559 19415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:20.544688 19415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:20.544726 19415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:20.544775 19415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:20.544811 19415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:20.550570 19415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:20.551860 19415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:20.552995 19415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:54:20.562430 19415 server_base.cc:1034] running on GCE node
W20260811 18:54:20.562367 19424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:20.562343 19421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:20.562343 19422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:20.563436 19415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:20.564792 19415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:20.566011 19415 hybrid_clock.cc:648] HybridClock initialized: now 1786474460565986 us; error 65 us; skew 500 ppm
Aug 11 18:54:20 dist-test-slave-9jjr krb5kdc[19401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:20.569141 19415 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:20.570298 19415 webserver.cc:460] Webserver started at http://127.0.0.1:43299/ using document root <none> and password file <none>
I20260811 18:54:20.570943 19415 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:20.571018 19415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:20.571208 19415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:20.573608 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "6e216ef70e894c089b58c6e114f0288b"
format_stamp: "Formatted at 2026-08-11 18:54:20 on dist-test-slave-9jjr"
server_key: "3f197dd5594ccab4e90633ae18323c69"
server_key_iv: "2fada7c20daad36d43ff28934324702f"
server_key_version: "encryptionkey@0"
I20260811 18:54:20.574334 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "6e216ef70e894c089b58c6e114f0288b"
format_stamp: "Formatted at 2026-08-11 18:54:20 on dist-test-slave-9jjr"
server_key: "3f197dd5594ccab4e90633ae18323c69"
server_key_iv: "2fada7c20daad36d43ff28934324702f"
server_key_version: "encryptionkey@0"
I20260811 18:54:20.578608 19415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:54:20.581409 19433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:20.582993 19415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:20.583166 19415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "6e216ef70e894c089b58c6e114f0288b"
format_stamp: "Formatted at 2026-08-11 18:54:20 on dist-test-slave-9jjr"
server_key: "3f197dd5594ccab4e90633ae18323c69"
server_key_iv: "2fada7c20daad36d43ff28934324702f"
server_key_version: "encryptionkey@0"
I20260811 18:54:20.583292 19415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:20.588398 19415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:20.592008 19415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:20.592235 19415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:20.600435 19415 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37981
I20260811 18:54:20.600456 19487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37981 every 8 connection(s)
I20260811 18:54:20.601511 19415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:20.604701 19488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:20.608064 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19415
I20260811 18:54:20.608259 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:20.608553 28111 external_mini_cluster.cc:1428] Setting key 153357ff7366e09ec32c198432181643
I20260811 18:54:20.611590 19488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b: Bootstrap starting.
I20260811 18:54:20.614693 19488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:20.615186 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:20.610335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43185 (local address 127.0.0.1:37981)
0811 18:54:20.610889 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:20.610908 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:20.610956 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:54:20.611879 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:20.611907 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:20.612302 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:20.612707 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:20.613605 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:20.613611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:20.613626 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:20.613639 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:20.613705 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:20.613731 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:20.613871 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:20.613941 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 18:54:20.614113 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":154,"threads_started":1}
I20260811 18:54:20.615660 19488 log.cc:824] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:20.618099 19488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b: No bootstrap required, opened a new log
I20260811 18:54:20.618362 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:20.622468 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:20.620677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59969 (local address 127.0.0.1:37981)
0811 18:54:20.620814 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:20.620818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:20.620922 (+ 104us) server_negotiation.cc:407] Connection header received
0811 18:54:20.621024 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:20.621027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:20.621069 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:20.621167 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:20.622016 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:20.622020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:20.622023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:20.622035 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:20.622066 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:20.622079 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:20.622237 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:20.622297 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:54:20.622341 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:54:20.622845 19488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } }
I20260811 18:54:20.623162 19488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:20.623245 19488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e216ef70e894c089b58c6e114f0288b, State: Initialized, Role: FOLLOWER
I20260811 18:54:20.623771 19488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } }
I20260811 18:54:20.623943 19488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:20.624039 19488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:20.624158 19488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:20.625253 19488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } }
I20260811 18:54:20.625666 19488 leader_election.cc:302] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e216ef70e894c089b58c6e114f0288b; no voters:
I20260811 18:54:20.626041 19488 leader_election.cc:288] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:20.626276 19498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:20.626566 19498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [term 1 LEADER]: Becoming Leader. State: Replica: 6e216ef70e894c089b58c6e114f0288b, State: Running, Role: LEADER
I20260811 18:54:20.626909 19498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } }
I20260811 18:54:20.627592 19488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:20.628770 19500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e216ef70e894c089b58c6e114f0288b. Latest consensus state: current_term: 1 leader_uuid: "6e216ef70e894c089b58c6e114f0288b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } } }
I20260811 18:54:20.628921 19500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:20.629241 19499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e216ef70e894c089b58c6e114f0288b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e216ef70e894c089b58c6e114f0288b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37981 } } }
I20260811 18:54:20.629334 19499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:20.629380 19509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:20.633077 19509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:20.641371 19509 catalog_manager.cc:1347] Generated new cluster ID: 7b6ca6da1ce6454ca8e1798878c88c7d
I20260811 18:54:20.641523 19509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:20.649842 19509 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:20.650352 19509 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:20.661664 19509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e216ef70e894c089b58c6e114f0288b: Generated new TSK 0
I20260811 18:54:20.662777 19509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:20.673936 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19415
2026-08-11T18:54:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:20 dist-test-slave-9jjr krb5kdc[19523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:20 dist-test-slave-9jjr krb5kdc[19523](info): set up 2 sockets
Aug 11 18:54:20 dist-test-slave-9jjr krb5kdc[19523](info): commencing operation
krb5kdc: starting...
W20260811 18:54:21.828531 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:21 dist-test-slave-9jjr krb5kdc[19523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:21.974435 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:22.094775 19539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:22.095218 19539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:22.095310 19539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:22.099499 19539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:22.099601 19539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:22.099635 19539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:22.099668 19539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:22.099704 19539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:22.105477 19539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:22.106740 19539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:22.107834 19539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:22.116981 19548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:22.116984 19544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:22.117040 19545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:22.117296 19539 server_base.cc:1034] running on GCE node
I20260811 18:54:22.118014 19539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:22.119320 19539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:22.120548 19539 hybrid_clock.cc:648] HybridClock initialized: now 1786474462120527 us; error 35 us; skew 500 ppm
Aug 11 18:54:22 dist-test-slave-9jjr krb5kdc[19523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:22.123962 19539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:22.125164 19539 webserver.cc:460] Webserver started at http://127.0.0.1:43149/ using document root <none> and password file <none>
I20260811 18:54:22.125896 19539 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:22.126010 19539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:22.126380 19539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:22.128836 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "f0eefc26e28146e18dedb9f97d5392ed"
format_stamp: "Formatted at 2026-08-11 18:54:22 on dist-test-slave-9jjr"
server_key: "8ed875c8a5c2935af04cef32055b749d"
server_key_iv: "4d61c2084b2e27ee6ece12bb4d4df174"
server_key_version: "encryptionkey@0"
I20260811 18:54:22.129487 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "f0eefc26e28146e18dedb9f97d5392ed"
format_stamp: "Formatted at 2026-08-11 18:54:22 on dist-test-slave-9jjr"
server_key: "8ed875c8a5c2935af04cef32055b749d"
server_key_iv: "4d61c2084b2e27ee6ece12bb4d4df174"
server_key_version: "encryptionkey@0"
I20260811 18:54:22.133970 19539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:54:22.136936 19555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:22.138571 19539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:22.138805 19539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "f0eefc26e28146e18dedb9f97d5392ed"
format_stamp: "Formatted at 2026-08-11 18:54:22 on dist-test-slave-9jjr"
server_key: "8ed875c8a5c2935af04cef32055b749d"
server_key_iv: "4d61c2084b2e27ee6ece12bb4d4df174"
server_key_version: "encryptionkey@0"
I20260811 18:54:22.138972 19539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:22.161753 19539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:22.166719 19539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:22.167023 19539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:22.176376 19539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44571
I20260811 18:54:22.176410 19609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44571 every 8 connection(s)
I20260811 18:54:22.177639 19539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:22.181018 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:22.183269 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19539
I20260811 18:54:22.183445 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:22.183745 28111 external_mini_cluster.cc:1428] Setting key a4f25fe28fe8b970da66c5182f715eb7
I20260811 18:54:22.189260 19610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed: Bootstrap starting.
I20260811 18:54:22.192400 19610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:22.193638 19610 log.cc:824] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:54:22 dist-test-slave-9jjr krb5kdc[19523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:54:22.196785 19610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed: No bootstrap required, opened a new log
I20260811 18:54:22.201038 19610 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } }
I20260811 18:54:22.201306 19610 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:22.201344 19610 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0eefc26e28146e18dedb9f97d5392ed, State: Initialized, Role: FOLLOWER
I20260811 18:54:22.201377 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:22.185566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:35055 (local address 127.0.0.1:44571)
0811 18:54:22.186319 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:22.186337 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:22.186379 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:54:22.187091 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:22.187109 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:22.187501 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:22.187836 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:22.188886 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:22.190014 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:22.191299 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:22.191822 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:22.191971 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:22.195787 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:22.195794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:22.195812 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:22.195848 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:22.197796 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:22.198567 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:22.198574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:22.198580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:22.198724 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:22.199142 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:22.199145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:22.199147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:22.199609 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:22.199920 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:22.200205 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:54:22.200468 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":249,"threads_started":1}
I20260811 18:54:22.201910 19610 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } }
I20260811 18:54:22.202104 19610 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:22.202260 19610 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:22.202409 19610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:22.203503 19610 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } }
I20260811 18:54:22.203898 19610 leader_election.cc:302] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0eefc26e28146e18dedb9f97d5392ed; no voters:
I20260811 18:54:22.204298 19610 leader_election.cc:288] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:22.204389 19616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:22.204579 19616 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [term 1 LEADER]: Becoming Leader. State: Replica: f0eefc26e28146e18dedb9f97d5392ed, State: Running, Role: LEADER
I20260811 18:54:22.204913 19616 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } }
I20260811 18:54:22.205196 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:22.205984 19610 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:22.206983 19618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0eefc26e28146e18dedb9f97d5392ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } } }
I20260811 18:54:22.207158 19618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:22.207504 19617 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eefc26e28146e18dedb9f97d5392ed. Latest consensus state: current_term: 1 leader_uuid: "f0eefc26e28146e18dedb9f97d5392ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eefc26e28146e18dedb9f97d5392ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44571 } } }
I20260811 18:54:22.207620 19617 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:22.207947 19631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:22.211841 19631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:22.219144 19631 catalog_manager.cc:1347] Generated new cluster ID: 46268d65ba7b459aa9ae1d2968facee9
I20260811 18:54:22.219236 19631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:22.220440 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:22.209148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:52497 (local address 127.0.0.1:44571)
0811 18:54:22.209296 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:22.209302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:22.209599 (+ 297us) server_negotiation.cc:407] Connection header received
0811 18:54:22.209750 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:22.209753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:22.209813 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:22.209910 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:22.210854 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:22.211689 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:22.213136 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:22.213542 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:22.213672 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:22.215721 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:22.215726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:22.215729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:22.215777 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:22.217828 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:22.218635 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:22.218639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:22.218641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:22.218745 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:22.219264 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:22.219270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:22.219274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:22.219536 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:22.219704 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:22.220136 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 18:54:22.220267 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:54:22.235693 19631 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:22.236740 19631 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:22.248124 19631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0eefc26e28146e18dedb9f97d5392ed: Generated new TSK 0
I20260811 18:54:22.248943 19631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:22.256932 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19539
W20260811 18:54:22.266793 19521 connection.cc:535] client connection to 127.0.0.1:44571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44571): Connection reset by peer (error 104)
2026-08-11T18:54:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:22 dist-test-slave-9jjr krb5kdc[19644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:22 dist-test-slave-9jjr krb5kdc[19644](info): set up 2 sockets
Aug 11 18:54:22 dist-test-slave-9jjr krb5kdc[19644](info): commencing operation
krb5kdc: starting...
W20260811 18:54:23.384529 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:23 dist-test-slave-9jjr krb5kdc[19644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:23.530164 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:23.650511 19659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:23.650821 19659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:23.650868 19659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:23.655088 19659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:23.655165 19659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:23.655185 19659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:23.655203 19659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:23.655220 19659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:23.661047 19659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:23.662400 19659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:23.663590 19659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:23.672649 19667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:23.672721 19664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:23.672871 19659 server_base.cc:1034] running on GCE node
W20260811 18:54:23.672649 19665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:23.673635 19659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:23.674908 19659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:23.676108 19659 hybrid_clock.cc:648] HybridClock initialized: now 1786474463676081 us; error 48 us; skew 500 ppm
Aug 11 18:54:23 dist-test-slave-9jjr krb5kdc[19644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474463, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:23.679468 19659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:23.680557 19659 webserver.cc:460] Webserver started at http://127.0.0.1:34831/ using document root <none> and password file <none>
I20260811 18:54:23.681243 19659 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:23.681318 19659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:23.681553 19659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:23.683882 19659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "1aa6210d88604569b2f186963b3be7d1"
format_stamp: "Formatted at 2026-08-11 18:54:23 on dist-test-slave-9jjr"
server_key: "943a72a9a0ca8f837391c325b9891674"
server_key_iv: "6962f2f03e06a0c26ca559fccc374997"
server_key_version: "encryptionkey@0"
I20260811 18:54:23.684475 19659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "1aa6210d88604569b2f186963b3be7d1"
format_stamp: "Formatted at 2026-08-11 18:54:23 on dist-test-slave-9jjr"
server_key: "943a72a9a0ca8f837391c325b9891674"
server_key_iv: "6962f2f03e06a0c26ca559fccc374997"
server_key_version: "encryptionkey@0"
I20260811 18:54:23.688757 19659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 18:54:23.691818 19674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:23.693372 19659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:54:23.693547 19659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "1aa6210d88604569b2f186963b3be7d1"
format_stamp: "Formatted at 2026-08-11 18:54:23 on dist-test-slave-9jjr"
server_key: "943a72a9a0ca8f837391c325b9891674"
server_key_iv: "6962f2f03e06a0c26ca559fccc374997"
server_key_version: "encryptionkey@0"
I20260811 18:54:23.693663 19659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:23.706305 19659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:23.709486 19659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:23.709659 19659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:23.717379 19659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43369
I20260811 18:54:23.717396 19729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43369 every 8 connection(s)
I20260811 18:54:23.718458 19659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:23.718864 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19659
I20260811 18:54:23.719040 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:23.719409 28111 external_mini_cluster.cc:1428] Setting key be1058838ae0a5a959bbe90f93a33c5e
I20260811 18:54:23.721890 19730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:23.729058 19730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: Bootstrap starting.
I20260811 18:54:23.729399 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:23.720856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:59535 (local address 127.0.0.1:43369)
0811 18:54:23.721312 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:23.721335 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:23.721372 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:54:23.722106 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:23.722171 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:23.722542 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:23.722850 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:23.723268 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:23.724303 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:23.725598 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:23.726207 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:23.726374 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:23.727429 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:23.727434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:23.727449 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:23.727894 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:23.727959 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:23.727985 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:23.728103 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:23.728331 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 18:54:23.728517 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":167,"threads_started":1}
I20260811 18:54:23.731863 19730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:23.732841 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:23.732841 19730 log.cc:824] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:23.735522 19730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: No bootstrap required, opened a new log
I20260811 18:54:23.740265 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:23.735245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:39305 (local address 127.0.0.1:43369)
0811 18:54:23.735369 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:23.735372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:23.735570 (+ 198us) server_negotiation.cc:407] Connection header received
0811 18:54:23.735736 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:23.735739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:23.735785 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:23.735864 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:23.736186 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:23.736746 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:23.738012 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:23.738322 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:23.738428 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:23.739536 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:23.739539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:23.739541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:23.739558 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:23.739590 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:23.739601 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:23.739684 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:23.739917 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 18:54:23.740009 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:54:23.740926 19730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260811 18:54:23.741204 19730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:23.741267 19730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa6210d88604569b2f186963b3be7d1, State: Initialized, Role: FOLLOWER
I20260811 18:54:23.741822 19730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260811 18:54:23.741962 19730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:23.742056 19730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:23.742192 19730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:23.743331 19730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260811 18:54:23.743738 19730 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa6210d88604569b2f186963b3be7d1; no voters:
I20260811 18:54:23.744114 19730 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:23.744320 19740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:23.744656 19740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 1aa6210d88604569b2f186963b3be7d1, State: Running, Role: LEADER
I20260811 18:54:23.745079 19740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260811 18:54:23.745574 19730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:23.746961 19742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aa6210d88604569b2f186963b3be7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } } }
I20260811 18:54:23.747108 19742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:23.747537 19743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa6210d88604569b2f186963b3be7d1. Latest consensus state: current_term: 1 leader_uuid: "1aa6210d88604569b2f186963b3be7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa6210d88604569b2f186963b3be7d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } } }
I20260811 18:54:23.747680 19743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1 [sys.catalog]: This master's current role is: LEADER
W20260811 18:54:23.751796 19761 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:23.751899 19761 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:23.751997 19756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:23.752843 19756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:23.759194 19756 catalog_manager.cc:1347] Generated new cluster ID: ed0408a1bc7944f088bc6a3195fcc052
I20260811 18:54:23.759296 19756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:23.767773 19756 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:23.769259 19756 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:23.780954 19756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aa6210d88604569b2f186963b3be7d1: Generated new TSK 0
I20260811 18:54:23.782073 19756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:23.789482 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19659
W20260811 18:54:23.802691 19642 connection.cc:535] client connection to 127.0.0.1:43369 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43369): Connection reset by peer (error 104)
2026-08-11T18:54:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:23 dist-test-slave-9jjr krb5kdc[19767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:23 dist-test-slave-9jjr krb5kdc[19767](info): set up 2 sockets
Aug 11 18:54:23 dist-test-slave-9jjr krb5kdc[19767](info): commencing operation
krb5kdc: starting...
W20260811 18:54:24.943981 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:25 dist-test-slave-9jjr krb5kdc[19767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:25.089555 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:25.215062 19780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:25.215467 19780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:25.215559 19780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:25.220067 19780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:25.220206 19780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:25.220245 19780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:25.220294 19780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:25.220336 19780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:25.227113 19780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:25.228569 19780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:25.229866 19780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:25.239737 19786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:25.239746 19788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:25.239730 19785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:25.240211 19780 server_base.cc:1034] running on GCE node
I20260811 18:54:25.240939 19780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:25.242214 19780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:25.243405 19780 hybrid_clock.cc:648] HybridClock initialized: now 1786474465243383 us; error 42 us; skew 500 ppm
Aug 11 18:54:25 dist-test-slave-9jjr krb5kdc[19767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:25.246615 19780 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:25.247625 19780 webserver.cc:460] Webserver started at http://127.0.0.1:42907/ using document root <none> and password file <none>
I20260811 18:54:25.248203 19780 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:25.248268 19780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:25.248464 19780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:25.250730 19780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "2cad1d982db744a5bb54e9c31ceb3093"
format_stamp: "Formatted at 2026-08-11 18:54:25 on dist-test-slave-9jjr"
server_key: "91a481025e648590259e1401a38ec17d"
server_key_iv: "a2abe5b1f4d88f76c39051efcf2d5a5a"
server_key_version: "encryptionkey@0"
I20260811 18:54:25.251329 19780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "2cad1d982db744a5bb54e9c31ceb3093"
format_stamp: "Formatted at 2026-08-11 18:54:25 on dist-test-slave-9jjr"
server_key: "91a481025e648590259e1401a38ec17d"
server_key_iv: "a2abe5b1f4d88f76c39051efcf2d5a5a"
server_key_version: "encryptionkey@0"
I20260811 18:54:25.256668 19780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:54:25.260110 19796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:25.261610 19780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:25.261758 19780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "2cad1d982db744a5bb54e9c31ceb3093"
format_stamp: "Formatted at 2026-08-11 18:54:25 on dist-test-slave-9jjr"
server_key: "91a481025e648590259e1401a38ec17d"
server_key_iv: "a2abe5b1f4d88f76c39051efcf2d5a5a"
server_key_version: "encryptionkey@0"
I20260811 18:54:25.261879 19780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:25.266778 19780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:25.269917 19780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:25.270192 19780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:25.277760 19780 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46351
I20260811 18:54:25.277791 19849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46351 every 8 connection(s)
I20260811 18:54:25.278820 19780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:25.281787 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:25.287740 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093: Bootstrap starting.
I20260811 18:54:25.288097 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19780
I20260811 18:54:25.288295 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:25.288615 28111 external_mini_cluster.cc:1428] Setting key bb8eab28744eafba0fb43e2b89a4eb57
I20260811 18:54:25.291054 19850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:25.292008 19850 log.cc:824] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:25.294098 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093: No bootstrap required, opened a new log
I20260811 18:54:25.295801 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:25.290159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:56231 (local address 127.0.0.1:46351)
0811 18:54:25.290636 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:25.290654 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:25.290697 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:54:25.291641 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:25.291666 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:25.292043 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:25.292434 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:25.294578 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:25.294583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:25.294595 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:25.294608 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:25.294667 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:25.294688 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:25.294842 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:25.294943 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 18:54:25.295110 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":115,"threads_started":1}
I20260811 18:54:25.298406 19850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } }
I20260811 18:54:25.298719 19850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:25.298763 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:25.298786 19850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cad1d982db744a5bb54e9c31ceb3093, State: Initialized, Role: FOLLOWER
I20260811 18:54:25.299253 19850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } }
I20260811 18:54:25.299414 19850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:25.299505 19850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:25.299605 19850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:25.300582 19850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } }
I20260811 18:54:25.300949 19850 leader_election.cc:302] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cad1d982db744a5bb54e9c31ceb3093; no voters:
I20260811 18:54:25.301324 19850 leader_election.cc:288] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:25.301543 19859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:25.301806 19859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [term 1 LEADER]: Becoming Leader. State: Replica: 2cad1d982db744a5bb54e9c31ceb3093, State: Running, Role: LEADER
I20260811 18:54:25.302240 19859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } }
I20260811 18:54:25.302917 19850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:25.303442 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:25.301255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:38523 (local address 127.0.0.1:46351)
0811 18:54:25.301407 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:25.301413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:25.301706 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:54:25.301919 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:25.301922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:25.301963 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:25.302014 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:25.302964 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:25.302968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:25.302970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:25.302982 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:25.303012 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:25.303023 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:25.303097 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:25.303283 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 18:54:25.303328 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:54:25.303973 19859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cad1d982db744a5bb54e9c31ceb3093. Latest consensus state: current_term: 1 leader_uuid: "2cad1d982db744a5bb54e9c31ceb3093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } } }
I20260811 18:54:25.304108 19859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:25.304549 19871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:25.304725 19861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cad1d982db744a5bb54e9c31ceb3093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cad1d982db744a5bb54e9c31ceb3093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46351 } } }
I20260811 18:54:25.304847 19861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:25.307941 19871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:25.313299 19871 catalog_manager.cc:1347] Generated new cluster ID: 4abde0d73d944ebbb8c41cf7a0e273d3
I20260811 18:54:25.313387 19871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:25.342360 19871 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:25.342718 19871 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:25.350456 19871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cad1d982db744a5bb54e9c31ceb3093: Generated new TSK 0
I20260811 18:54:25.351207 19871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:25.362447 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19780
2026-08-11T18:54:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:25 dist-test-slave-9jjr krb5kdc[19886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:25 dist-test-slave-9jjr krb5kdc[19886](info): set up 2 sockets
Aug 11 18:54:25 dist-test-slave-9jjr krb5kdc[19886](info): commencing operation
krb5kdc: starting...
W20260811 18:54:26.505540 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:26 dist-test-slave-9jjr krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:26.654793 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:26.777319 19901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:26.777647 19901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:26.777693 19901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:26.782421 19901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:26.782503 19901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:26.782526 19901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:26.782543 19901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:26.782559 19901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:26.790377 19901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:26.791647 19901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:26.792860 19901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:26.802915 19906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:26.802915 19907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:26.803232 19901 server_base.cc:1034] running on GCE node
W20260811 18:54:26.803442 19909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:26.804165 19901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:26.805466 19901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:26.806639 19901 hybrid_clock.cc:648] HybridClock initialized: now 1786474466806609 us; error 51 us; skew 500 ppm
Aug 11 18:54:26 dist-test-slave-9jjr krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474466, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:26.814961 19901 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:26.816167 19901 webserver.cc:460] Webserver started at http://127.0.0.1:35415/ using document root <none> and password file <none>
I20260811 18:54:26.816810 19901 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:26.816879 19901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:26.817142 19901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:26.819518 19901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "950da7e7667f419cb6a11ce71604f66d"
format_stamp: "Formatted at 2026-08-11 18:54:26 on dist-test-slave-9jjr"
server_key: "8411475e2c91916c86f8d2633c4b9c07"
server_key_iv: "5e1b650bc729b731b0bd77834520fae3"
server_key_version: "encryptionkey@0"
I20260811 18:54:26.820158 19901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "950da7e7667f419cb6a11ce71604f66d"
format_stamp: "Formatted at 2026-08-11 18:54:26 on dist-test-slave-9jjr"
server_key: "8411475e2c91916c86f8d2633c4b9c07"
server_key_iv: "5e1b650bc729b731b0bd77834520fae3"
server_key_version: "encryptionkey@0"
I20260811 18:54:26.824517 19901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:26.827405 19916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:26.828910 19901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:26.829118 19901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "950da7e7667f419cb6a11ce71604f66d"
format_stamp: "Formatted at 2026-08-11 18:54:26 on dist-test-slave-9jjr"
server_key: "8411475e2c91916c86f8d2633c4b9c07"
server_key_iv: "5e1b650bc729b731b0bd77834520fae3"
server_key_version: "encryptionkey@0"
I20260811 18:54:26.829262 19901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:26.847980 19901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:26.852061 19901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:26.852322 19901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:26.863569 19901 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39955
I20260811 18:54:26.863591 19972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39955 every 8 connection(s)
I20260811 18:54:26.864791 19901 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:26.868288 19973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:26.874338 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 19901
I20260811 18:54:26.874547 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:26.874866 28111 external_mini_cluster.cc:1428] Setting key ae3b6d7406bbbb46acd2f8491661b62d
I20260811 18:54:26.876196 19973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d: Bootstrap starting.
I20260811 18:54:26.879176 19973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:26.880304 19973 log.cc:824] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:26.883088 19973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d: No bootstrap required, opened a new log
Aug 11 18:54:26 dist-test-slave-9jjr krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:54:26.888077 19973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260811 18:54:26.888824 19973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:26.888968 19973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950da7e7667f419cb6a11ce71604f66d, State: Initialized, Role: FOLLOWER
I20260811 18:54:26.889565 19973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260811 18:54:26.889770 19973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:26.889873 19973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:26.890025 19973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:26.891227 19973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260811 18:54:26.891665 19973 leader_election.cc:302] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 950da7e7667f419cb6a11ce71604f66d; no voters:
I20260811 18:54:26.892091 19973 leader_election.cc:288] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:26.892398 19978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:26.892728 19978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [term 1 LEADER]: Becoming Leader. State: Replica: 950da7e7667f419cb6a11ce71604f66d, State: Running, Role: LEADER
I20260811 18:54:26.893101 19978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260811 18:54:26.893337 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:26.876767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:52963 (local address 127.0.0.1:39955)
0811 18:54:26.877334 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:26.877363 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:26.877406 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:54:26.878279 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:26.878303 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:26.878726 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:26.879057 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:26.880194 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:26.881600 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:26.882873 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:26.883493 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:26.883654 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:26.886922 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:26.886928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:26.886948 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:26.886986 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:26.889260 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:26.890132 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:26.890167 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:26.890176 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:26.890376 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:26.890793 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:26.890796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:26.890798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:26.891302 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:26.891587 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:26.891932 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 18:54:26.892243 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":190,"threads_started":1}
I20260811 18:54:26.893689 19973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:26.894963 19979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "950da7e7667f419cb6a11ce71604f66d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } } }
I20260811 18:54:26.895140 19979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:26.895596 19987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:26.895553 19980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950da7e7667f419cb6a11ce71604f66d. Latest consensus state: current_term: 1 leader_uuid: "950da7e7667f419cb6a11ce71604f66d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950da7e7667f419cb6a11ce71604f66d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } } }
I20260811 18:54:26.896224 19980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:26.897096 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:26.899287 19987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:26.906709 19987 catalog_manager.cc:1347] Generated new cluster ID: 038f3d5ce7b34a51b3e342eb2bed5ce1
I20260811 18:54:26.906797 19987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:26.909597 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:26.899800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:44439 (local address 127.0.0.1:39955)
0811 18:54:26.899927 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:26.899933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:26.900297 (+ 364us) server_negotiation.cc:407] Connection header received
0811 18:54:26.900462 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:26.900466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:26.900523 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:26.900616 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:26.901344 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:26.901985 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:26.903265 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:26.903597 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:26.903724 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:26.905473 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:26.905477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:26.905480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:26.905516 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:26.907323 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:26.907994 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:26.907997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:26.907999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:26.908086 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:26.908463 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:26.908466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:26.908468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:26.908678 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:26.908807 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:26.909140 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 18:54:26.909249 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:54:26.924823 19987 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:26.926290 19987 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:26.937400 19987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 950da7e7667f419cb6a11ce71604f66d: Generated new TSK 0
I20260811 18:54:26.938364 19987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:26.947616 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 19901
W20260811 18:54:26.956348 19882 connection.cc:535] client connection to 127.0.0.1:39955 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39955): Connection reset by peer (error 104)
2026-08-11T18:54:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:27 dist-test-slave-9jjr krb5kdc[20004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:27 dist-test-slave-9jjr krb5kdc[20004](info): set up 2 sockets
Aug 11 18:54:27 dist-test-slave-9jjr krb5kdc[20004](info): commencing operation
krb5kdc: starting...
W20260811 18:54:28.069144 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:28 dist-test-slave-9jjr krb5kdc[20004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:28.214346 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:28.337105 20019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:28.337430 20019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:28.337476 20019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:28.341791 20019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:28.341876 20019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:28.341895 20019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:28.341914 20019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:28.341929 20019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:28.348102 20019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:28.349630 20019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:28.350957 20019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:28.361315 20029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:28.361315 20026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:28.361320 20027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:28.361994 20019 server_base.cc:1034] running on GCE node
I20260811 18:54:28.362766 20019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:28.364039 20019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:28.365257 20019 hybrid_clock.cc:648] HybridClock initialized: now 1786474468365226 us; error 49 us; skew 500 ppm
Aug 11 18:54:28 dist-test-slave-9jjr krb5kdc[20004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:28.368548 20019 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:28.369669 20019 webserver.cc:460] Webserver started at http://127.0.0.1:36229/ using document root <none> and password file <none>
I20260811 18:54:28.370359 20019 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:28.370457 20019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:28.370708 20019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:28.373023 20019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "bed1e77a0aa3474da1304d5fb50d264b"
format_stamp: "Formatted at 2026-08-11 18:54:28 on dist-test-slave-9jjr"
server_key: "aa290a497e7d787595dbbd641c95bd4f"
server_key_iv: "12568b261d12cc8dfa6775f10764ae27"
server_key_version: "encryptionkey@0"
I20260811 18:54:28.373618 20019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "bed1e77a0aa3474da1304d5fb50d264b"
format_stamp: "Formatted at 2026-08-11 18:54:28 on dist-test-slave-9jjr"
server_key: "aa290a497e7d787595dbbd641c95bd4f"
server_key_iv: "12568b261d12cc8dfa6775f10764ae27"
server_key_version: "encryptionkey@0"
I20260811 18:54:28.377734 20019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 18:54:28.380668 20036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:28.382221 20019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:28.382400 20019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "bed1e77a0aa3474da1304d5fb50d264b"
format_stamp: "Formatted at 2026-08-11 18:54:28 on dist-test-slave-9jjr"
server_key: "aa290a497e7d787595dbbd641c95bd4f"
server_key_iv: "12568b261d12cc8dfa6775f10764ae27"
server_key_version: "encryptionkey@0"
I20260811 18:54:28.382540 20019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:28.397511 20019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:28.401180 20019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:28.401423 20019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:28.410104 20019 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34143
I20260811 18:54:28.410166 20090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34143 every 8 connection(s)
I20260811 18:54:28.411202 20019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:28.413066 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20019
I20260811 18:54:28.413226 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:28.413551 28111 external_mini_cluster.cc:1428] Setting key 800320635457525fbff1974e36bf9765
I20260811 18:54:28.414680 20092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:28.420787 20092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b: Bootstrap starting.
I20260811 18:54:28.423306 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:28.415245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:37727 (local address 127.0.0.1:34143)
0811 18:54:28.415723 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:28.415738 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:28.415791 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:54:28.416443 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:28.416460 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:28.416818 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:28.417162 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:28.417601 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:28.418721 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:28.419659 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:28.420157 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:28.420332 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:28.421395 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:28.421400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:28.421413 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:28.421825 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:28.421886 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:28.421909 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:28.422016 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:28.422290 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:54:28.422460 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":146,"threads_started":1}
I20260811 18:54:28.423579 20092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:28.424386 20092 log.cc:824] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:28.426710 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:28.427022 20092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b: No bootstrap required, opened a new log
I20260811 18:54:28.431627 20092 raft_consensus.cc:348] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260811 18:54:28.431917 20092 raft_consensus.cc:374] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:28.431953 20092 raft_consensus.cc:729] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed1e77a0aa3474da1304d5fb50d264b, State: Initialized, Role: FOLLOWER
I20260811 18:54:28.432533 20092 consensus_queue.cc:260] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260811 18:54:28.432713 20092 raft_consensus.cc:388] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:28.432812 20092 raft_consensus.cc:482] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:28.432933 20092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:28.433683 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:28.428971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:37319 (local address 127.0.0.1:34143)
0811 18:54:28.429121 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:28.429127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:28.429352 (+ 225us) server_negotiation.cc:407] Connection header received
0811 18:54:28.429507 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:28.429510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:28.429553 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:28.429614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:28.429942 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:28.430541 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:28.431643 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:28.431889 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:28.432045 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:28.433102 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:28.433106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:28.433116 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:28.433139 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:28.433169 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:28.433182 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:28.433257 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:28.433449 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 18:54:28.433510 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:54:28.434418 20092 raft_consensus.cc:504] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260811 18:54:28.434865 20092 leader_election.cc:302] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bed1e77a0aa3474da1304d5fb50d264b; no voters:
I20260811 18:54:28.435261 20092 leader_election.cc:288] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:28.435446 20105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:28.435745 20105 raft_consensus.cc:686] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [term 1 LEADER]: Becoming Leader. State: Replica: bed1e77a0aa3474da1304d5fb50d264b, State: Running, Role: LEADER
I20260811 18:54:28.436169 20105 consensus_queue.cc:237] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260811 18:54:28.436759 20092 sys_catalog.cc:564] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:28.438005 20107 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bed1e77a0aa3474da1304d5fb50d264b. Latest consensus state: current_term: 1 leader_uuid: "bed1e77a0aa3474da1304d5fb50d264b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } } }
I20260811 18:54:28.438194 20107 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:28.438702 20115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:28.438966 20106 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bed1e77a0aa3474da1304d5fb50d264b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed1e77a0aa3474da1304d5fb50d264b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } } }
I20260811 18:54:28.439147 20106 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:28.442373 20115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:28.449129 20115 catalog_manager.cc:1347] Generated new cluster ID: 4fcc20bd83a54e6092cccc5ed66e5e90
I20260811 18:54:28.449290 20115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:28.463749 20115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:28.464807 20115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:28.476451 20115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bed1e77a0aa3474da1304d5fb50d264b: Generated new TSK 0
I20260811 18:54:28.477398 20115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:28.487620 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20019
W20260811 18:54:28.497246 20002 connection.cc:535] client connection to 127.0.0.1:34143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34143): Connection reset by peer (error 104)
2026-08-11T18:54:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:28 dist-test-slave-9jjr krb5kdc[20128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:28 dist-test-slave-9jjr krb5kdc[20128](info): set up 2 sockets
Aug 11 18:54:28 dist-test-slave-9jjr krb5kdc[20128](info): commencing operation
krb5kdc: starting...
W20260811 18:54:29.603894 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:29 dist-test-slave-9jjr krb5kdc[20128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:54:29.750021 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:29.872167 20141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:29.872572 20141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:29.872665 20141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:29.877091 20141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:29.877209 20141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:29.877249 20141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:29.877318 20141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:29.877359 20141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:29.883178 20141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:29.884454 20141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:29.885546 20141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:29.894591 20148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:29.894559 20151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:29.894688 20141 server_base.cc:1034] running on GCE node
W20260811 18:54:29.894568 20147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:29.895615 20141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:29.896780 20141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:29.897957 20141 hybrid_clock.cc:648] HybridClock initialized: now 1786474469897926 us; error 46 us; skew 500 ppm
Aug 11 18:54:29 dist-test-slave-9jjr krb5kdc[20128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474469, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:29.901099 20141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:54:29.902029 20141 webserver.cc:460] Webserver started at http://127.0.0.1:38879/ using document root <none> and password file <none>
I20260811 18:54:29.902652 20141 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:29.902719 20141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:29.902891 20141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:29.905050 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "6fee1385a25241b187c27c87e1c809fe"
format_stamp: "Formatted at 2026-08-11 18:54:29 on dist-test-slave-9jjr"
server_key: "fae5cbad674639f7c36d30a5c2c4f7af"
server_key_iv: "56c3e45675b234085833033d6c278fb1"
server_key_version: "encryptionkey@0"
I20260811 18:54:29.905614 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "6fee1385a25241b187c27c87e1c809fe"
format_stamp: "Formatted at 2026-08-11 18:54:29 on dist-test-slave-9jjr"
server_key: "fae5cbad674639f7c36d30a5c2c4f7af"
server_key_iv: "56c3e45675b234085833033d6c278fb1"
server_key_version: "encryptionkey@0"
I20260811 18:54:29.909583 20141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:54:29.912236 20160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:29.913550 20141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:54:29.913816 20141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "6fee1385a25241b187c27c87e1c809fe"
format_stamp: "Formatted at 2026-08-11 18:54:29 on dist-test-slave-9jjr"
server_key: "fae5cbad674639f7c36d30a5c2c4f7af"
server_key_iv: "56c3e45675b234085833033d6c278fb1"
server_key_version: "encryptionkey@0"
I20260811 18:54:29.913949 20141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:29.919633 20141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:29.923712 20141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:29.923962 20141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:29.932204 20141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37905
I20260811 18:54:29.932309 20215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37905 every 8 connection(s)
I20260811 18:54:29.933300 20141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:29.936462 20216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:29.938393 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20141
I20260811 18:54:29.938548 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:29.938889 28111 external_mini_cluster.cc:1428] Setting key d0cfe1874d6c13dde9471a8fe8eedd85
I20260811 18:54:29.944489 20216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe: Bootstrap starting.
I20260811 18:54:29.945501 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:29.940768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:51525 (local address 127.0.0.1:37905)
0811 18:54:29.941381 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:29.941398 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:29.941442 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:54:29.942264 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:29.942313 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:29.942676 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:29.943112 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:29.944091 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:29.944096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:29.944110 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:29.944122 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:29.944187 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:29.944210 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:29.944391 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:29.944470 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:54:29.944680 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":200,"threads_started":1}
I20260811 18:54:29.947785 20216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:29.948810 20216 log.cc:824] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:29.949136 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:29.951397 20216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe: No bootstrap required, opened a new log
I20260811 18:54:29.953570 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:29.951475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:54681 (local address 127.0.0.1:37905)
0811 18:54:29.951612 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:29.951619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:29.951851 (+ 232us) server_negotiation.cc:407] Connection header received
0811 18:54:29.952035 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:29.952038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:29.952077 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:29.952173 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:29.953057 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:29.953060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:29.953062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:29.953079 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:29.953107 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:29.953119 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:29.953203 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:29.953321 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 18:54:29.953364 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:54:29.956413 20216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260811 18:54:29.956744 20216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:29.956832 20216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fee1385a25241b187c27c87e1c809fe, State: Initialized, Role: FOLLOWER
I20260811 18:54:29.957583 20216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260811 18:54:29.957919 20216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:29.958024 20216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:29.958165 20216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:29.959338 20216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260811 18:54:29.959759 20216 leader_election.cc:302] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fee1385a25241b187c27c87e1c809fe; no voters:
I20260811 18:54:29.960309 20216 leader_election.cc:288] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:29.960667 20227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:29.961002 20227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [term 1 LEADER]: Becoming Leader. State: Replica: 6fee1385a25241b187c27c87e1c809fe, State: Running, Role: LEADER
I20260811 18:54:29.961426 20227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } }
I20260811 18:54:29.962038 20216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:29.963356 20229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fee1385a25241b187c27c87e1c809fe. Latest consensus state: current_term: 1 leader_uuid: "6fee1385a25241b187c27c87e1c809fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } } }
I20260811 18:54:29.963510 20229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:29.963893 20228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fee1385a25241b187c27c87e1c809fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fee1385a25241b187c27c87e1c809fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37905 } } }
I20260811 18:54:29.964073 20228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:29.965272 20237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:29.968746 20237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:29.975139 20237 catalog_manager.cc:1347] Generated new cluster ID: 4eafb6836a1e4f599bd872ab5d2d2920
I20260811 18:54:29.975241 20237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:29.996491 20237 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:29.996951 20237 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:30.002478 20237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fee1385a25241b187c27c87e1c809fe: Generated new TSK 0
I20260811 18:54:30.003433 20237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:30.014387 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20141
2026-08-11T18:54:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:30 dist-test-slave-9jjr krb5kdc[20252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:30 dist-test-slave-9jjr krb5kdc[20252](info): set up 2 sockets
Aug 11 18:54:30 dist-test-slave-9jjr krb5kdc[20252](info): commencing operation
krb5kdc: starting...
W20260811 18:54:31.140682 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:31.278573 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37655
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:31.401211 20268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:31.401563 20268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:31.401659 20268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:31.406075 20268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:31.406211 20268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:31.406255 20268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:31.406294 20268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:31.406329 20268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:31.411942 20268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:31.413189 20268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:31.414373 20268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:31.423553 20276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:31.423563 20273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:31.423565 20274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:31.423632 20268 server_base.cc:1034] running on GCE node
I20260811 18:54:31.424467 20268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:31.425637 20268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:31.426826 20268 hybrid_clock.cc:648] HybridClock initialized: now 1786474471426792 us; error 49 us; skew 500 ppm
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474471, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:31.429817 20268 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:31.430866 20268 webserver.cc:460] Webserver started at http://127.27.115.254:42969/ using document root <none> and password file <none>
I20260811 18:54:31.431468 20268 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:31.431561 20268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:31.431818 20268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:31.434093 20268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "6e9285ddabcb45eda5d059f9e3da77d6"
format_stamp: "Formatted at 2026-08-11 18:54:31 on dist-test-slave-9jjr"
server_key: "e40dccbcb328097759edd58b656e5efe"
server_key_iv: "a196a83760892684409dc11ca2d362b4"
server_key_version: "encryptionkey@0"
I20260811 18:54:31.434806 20268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "6e9285ddabcb45eda5d059f9e3da77d6"
format_stamp: "Formatted at 2026-08-11 18:54:31 on dist-test-slave-9jjr"
server_key: "e40dccbcb328097759edd58b656e5efe"
server_key_iv: "a196a83760892684409dc11ca2d362b4"
server_key_version: "encryptionkey@0"
I20260811 18:54:31.438975 20268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:31.441572 20283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:31.443048 20268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:54:31.443228 20268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "6e9285ddabcb45eda5d059f9e3da77d6"
format_stamp: "Formatted at 2026-08-11 18:54:31 on dist-test-slave-9jjr"
server_key: "e40dccbcb328097759edd58b656e5efe"
server_key_iv: "a196a83760892684409dc11ca2d362b4"
server_key_version: "encryptionkey@0"
I20260811 18:54:31.443359 20268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:31.462311 20268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:31.465981 20268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:31.466272 20268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:31.474037 20268 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37655
I20260811 18:54:31.474061 20341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37655 every 8 connection(s)
I20260811 18:54:31.475140 20268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:31.477098 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20268
I20260811 18:54:31.477259 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:31.477579 28111 external_mini_cluster.cc:1428] Setting key ce27e6969902235d73c7ffa14f4474d4
I20260811 18:54:31.478888 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:31.484882 20342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: Bootstrap starting.
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:31.487694 20342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:31.488492 20342 log.cc:824] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:31.490885 20342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: No bootstrap required, opened a new log
I20260811 18:54:31.492843 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:31.479163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39655 (local address 127.27.115.254:37655)
0811 18:54:31.479647 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:31.479663 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:31.479701 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:54:31.480367 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:31.480384 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:31.480729 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:31.481072 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:31.481925 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:31.483017 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:31.483914 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:31.484397 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:31.484580 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:31.487646 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:31.487652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:31.487668 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:31.487703 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:31.489612 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:31.490243 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:31.490249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:31.490255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:31.490376 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:31.490767 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:31.490770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:31.490771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:31.491259 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:31.491492 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:31.491846 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 18:54:31.492088 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":160,"threads_started":1}
I20260811 18:54:31.495080 20342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } }
I20260811 18:54:31.495412 20342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:31.495494 20342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9285ddabcb45eda5d059f9e3da77d6, State: Initialized, Role: FOLLOWER
I20260811 18:54:31.496048 20342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } }
I20260811 18:54:31.496254 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:31.496261 20342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:31.496358 20342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:31.496467 20342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:31.497627 20342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } }
I20260811 18:54:31.498076 20342 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9285ddabcb45eda5d059f9e3da77d6; no voters:
I20260811 18:54:31.498536 20342 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:31.498657 20351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:31.498956 20351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6e9285ddabcb45eda5d059f9e3da77d6, State: Running, Role: LEADER
I20260811 18:54:31.499437 20351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } }
I20260811 18:54:31.500211 20342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:31.500948 20351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9285ddabcb45eda5d059f9e3da77d6. Latest consensus state: current_term: 1 leader_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } } }
I20260811 18:54:31.501740 20351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:31.501617 20353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9285ddabcb45eda5d059f9e3da77d6" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37655 } } }
I20260811 18:54:31.502075 20353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6 [sys.catalog]: This master's current role is: LEADER
W20260811 18:54:31.506094 20368 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:31.506294 20368 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:31.506429 20363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:31.507591 20363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:31.509598 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:31.498713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33779 (local address 127.27.115.254:37655)
0811 18:54:31.498990 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:31.498996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:31.499283 (+ 287us) server_negotiation.cc:407] Connection header received
0811 18:54:31.499428 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:31.499431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:31.499492 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:31.499590 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:31.500598 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:31.501464 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:31.502986 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:31.503365 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:31.503478 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:31.505376 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:31.505380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:31.505383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:31.505417 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:31.507349 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:31.508063 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:31.508067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:31.508069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:31.508168 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:31.508567 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:31.508570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:31.508572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:31.508807 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:31.508959 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:31.509300 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:54:31.509412 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260811 18:54:31.514793 20363 catalog_manager.cc:1347] Generated new cluster ID: bf92e3094edb4ad08c5fb12598ddb08a
I20260811 18:54:31.514894 20363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:31.528405 20363 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:31.529531 20363 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:31.544077 20363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9285ddabcb45eda5d059f9e3da77d6: Generated new TSK 0
I20260811 18:54:31.544945 20363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:31.557006 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20268
W20260811 18:54:31.566455 20250 connection.cc:535] client connection to 127.27.115.254:37655 recv error: Network error: failed to read from TLS socket (remote: 127.27.115.254:37655): Connection reset by peer (error 104)
2026-08-11T18:54:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20374](info): set up 2 sockets
Aug 11 18:54:31 dist-test-slave-9jjr krb5kdc[20374](info): commencing operation
krb5kdc: starting...
W20260811 18:54:32.707005 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:32 dist-test-slave-9jjr krb5kdc[20374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:32.850368 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36709
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:32.980638 20389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:32.980952 20389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:32.980997 20389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:32.985348 20389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:32.985446 20389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:32.985466 20389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:32.985487 20389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:32.985502 20389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:32.991266 20389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:36709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:32.992523 20389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:32.993667 20389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:33.003244 20396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:33.003278 20400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:33.003398 20389 server_base.cc:1034] running on GCE node
W20260811 18:54:33.003244 20397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:33.004267 20389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:33.005523 20389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:33.006719 20389 hybrid_clock.cc:648] HybridClock initialized: now 1786474473006703 us; error 34 us; skew 500 ppm
Aug 11 18:54:33 dist-test-slave-9jjr krb5kdc[20374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474473, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:33.026530 20389 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:33.027823 20389 webserver.cc:460] Webserver started at http://127.27.115.254:36743/ using document root <none> and password file <none>
I20260811 18:54:33.028445 20389 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:33.028509 20389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:33.028761 20389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:33.031044 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "fa7b21997de6426f9876c970f847987a"
format_stamp: "Formatted at 2026-08-11 18:54:33 on dist-test-slave-9jjr"
server_key: "1014788b29db40265f229718432c2a09"
server_key_iv: "bf80fd00f58a7b5b3fd2c8024bfc9054"
server_key_version: "encryptionkey@0"
I20260811 18:54:33.031672 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "fa7b21997de6426f9876c970f847987a"
format_stamp: "Formatted at 2026-08-11 18:54:33 on dist-test-slave-9jjr"
server_key: "1014788b29db40265f229718432c2a09"
server_key_iv: "bf80fd00f58a7b5b3fd2c8024bfc9054"
server_key_version: "encryptionkey@0"
I20260811 18:54:33.035807 20389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:33.038537 20407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:33.039985 20389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:33.040184 20389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "fa7b21997de6426f9876c970f847987a"
format_stamp: "Formatted at 2026-08-11 18:54:33 on dist-test-slave-9jjr"
server_key: "1014788b29db40265f229718432c2a09"
server_key_iv: "bf80fd00f58a7b5b3fd2c8024bfc9054"
server_key_version: "encryptionkey@0"
I20260811 18:54:33.040331 20389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:33.059876 20389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:33.063648 20389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:33.063915 20389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:33.072386 20389 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:36709
I20260811 18:54:33.072413 20463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:36709 every 8 connection(s)
I20260811 18:54:33.073493 20389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:33.076679 20464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:33.080041 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20389
I20260811 18:54:33.080211 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:33.080541 28111 external_mini_cluster.cc:1428] Setting key 3a3e52a103f16a0c7508bd3269060023
I20260811 18:54:33.083527 20464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a: Bootstrap starting.
I20260811 18:54:33.085958 20464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:33.086808 20464 log.cc:824] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:33.089311 20464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a: No bootstrap required, opened a new log
I20260811 18:54:33.090636 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:33.082209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55463 (local address 127.27.115.254:36709)
0811 18:54:33.082719 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:33.082737 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:33.082776 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:54:33.083491 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:33.083511 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:33.083949 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:33.084323 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:33.084760 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:33.085765 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:33.086824 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:33.087489 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:33.087679 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:33.088662 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:33.088668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:33.088682 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:33.089091 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:33.089187 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:33.089213 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:33.089329 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:33.089551 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 18:54:33.089738 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":150,"threads_started":1}
I20260811 18:54:33.094431 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:33.094210 20464 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } }
I20260811 18:54:33.094537 20464 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:33.094596 20464 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7b21997de6426f9876c970f847987a, State: Initialized, Role: FOLLOWER
I20260811 18:54:33.095134 20464 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } }
I20260811 18:54:33.095328 20464 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:33.095407 20464 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:33.095523 20464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:33.096946 20464 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } }
I20260811 18:54:33.097414 20464 leader_election.cc:302] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa7b21997de6426f9876c970f847987a; no voters:
I20260811 18:54:33.097813 20464 leader_election.cc:288] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:33.097887 20474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:33.098157 20474 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [term 1 LEADER]: Becoming Leader. State: Replica: fa7b21997de6426f9876c970f847987a, State: Running, Role: LEADER
I20260811 18:54:33.098609 20474 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } }
I20260811 18:54:33.099515 20464 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:33.100422 20475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa7b21997de6426f9876c970f847987a. Latest consensus state: current_term: 1 leader_uuid: "fa7b21997de6426f9876c970f847987a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } } }
I20260811 18:54:33.100564 20475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:33.100924 20476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa7b21997de6426f9876c970f847987a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b21997de6426f9876c970f847987a" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36709 } } }
I20260811 18:54:33.101017 20476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:33.102448 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:33.103365 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:33.097109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60243 (local address 127.27.115.254:36709)
0811 18:54:33.097242 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:33.097247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:33.097600 (+ 353us) server_negotiation.cc:407] Connection header received
0811 18:54:33.097806 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:33.097809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:33.097890 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:33.097970 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:33.098441 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:33.099230 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:33.100481 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:33.100895 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:33.100997 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:33.102697 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:33.102700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:33.102702 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:33.102717 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:33.102743 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:33.102753 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:33.102823 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:33.103051 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 18:54:33.103123 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":54}
I20260811 18:54:33.106019 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:33.113960 20484 catalog_manager.cc:1347] Generated new cluster ID: 5083da0039d04225ad707427b5ea627d
I20260811 18:54:33.114111 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:33.125975 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:33.127146 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:33.135712 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa7b21997de6426f9876c970f847987a: Generated new TSK 0
I20260811 18:54:33.136657 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:33.144134 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20389
W20260811 18:54:33.155731 20372 connection.cc:535] client connection to 127.27.115.254:36709 recv error: Network error: failed to read from TLS socket (remote: 127.27.115.254:36709): Connection reset by peer (error 104)
2026-08-11T18:54:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:33 dist-test-slave-9jjr krb5kdc[20498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:33 dist-test-slave-9jjr krb5kdc[20498](info): set up 2 sockets
Aug 11 18:54:33 dist-test-slave-9jjr krb5kdc[20498](info): commencing operation
krb5kdc: starting...
W20260811 18:54:34.293982 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:34 dist-test-slave-9jjr krb5kdc[20498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:34.456154 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33091
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:44877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:33091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:34.584502 20512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:34.584908 20512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:34.585153 20512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:34.590590 20512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:34.590742 20512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:34.590783 20512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:34.590826 20512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:34.590865 20512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:34.597354 20512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:44877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:33091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:33091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:34.598867 20512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:34.600167 20512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:34.610335 20518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:34.610319 20517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:34.610318 20520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:34.610373 20512 server_base.cc:1034] running on GCE node
I20260811 18:54:34.611384 20512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:34.612895 20512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:34.614187 20512 hybrid_clock.cc:648] HybridClock initialized: now 1786474474614098 us; error 53 us; skew 500 ppm
Aug 11 18:54:34 dist-test-slave-9jjr krb5kdc[20498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474474, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:34.617571 20512 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:34.618763 20512 webserver.cc:460] Webserver started at http://127.27.115.254:43833/ using document root <none> and password file <none>
I20260811 18:54:34.619413 20512 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:34.619516 20512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:34.619779 20512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:34.622066 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "51c251c0b9d440baa37b56a1206deded"
format_stamp: "Formatted at 2026-08-11 18:54:34 on dist-test-slave-9jjr"
server_key: "d52b479da97b2c438fee8059d95ca805"
server_key_iv: "5454227b02321b30965ddedfa1f42917"
server_key_version: "encryptionkey@0"
I20260811 18:54:34.622782 20512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "51c251c0b9d440baa37b56a1206deded"
format_stamp: "Formatted at 2026-08-11 18:54:34 on dist-test-slave-9jjr"
server_key: "d52b479da97b2c438fee8059d95ca805"
server_key_iv: "5454227b02321b30965ddedfa1f42917"
server_key_version: "encryptionkey@0"
I20260811 18:54:34.627053 20512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:34.630520 20527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:34.631981 20512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:54:34.632225 20512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "51c251c0b9d440baa37b56a1206deded"
format_stamp: "Formatted at 2026-08-11 18:54:34 on dist-test-slave-9jjr"
server_key: "d52b479da97b2c438fee8059d95ca805"
server_key_iv: "5454227b02321b30965ddedfa1f42917"
server_key_version: "encryptionkey@0"
I20260811 18:54:34.632370 20512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:34.639631 20512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:34.645821 20512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:34.646085 20512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:34.658268 20512 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:33091
I20260811 18:54:34.658330 20584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:33091 every 8 connection(s)
I20260811 18:54:34.660311 20512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:34.664285 20585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:34.666368 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20512
I20260811 18:54:34.666533 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:34.666853 28111 external_mini_cluster.cc:1428] Setting key ff016db783510669a5c4aa73f376822f
I20260811 18:54:34.673727 20585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded: Bootstrap starting.
I20260811 18:54:34.674672 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:34.668473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52251 (local address 127.27.115.254:33091)
0811 18:54:34.669033 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:34.669055 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:34.669628 (+ 573us) server_negotiation.cc:407] Connection header received
0811 18:54:34.670588 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:34.670611 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:34.670944 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:34.671329 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:34.672481 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:34.672489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:34.672513 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:34.672538 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:34.672681 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:34.672716 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:34.672978 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:34.673164 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 18:54:34.673636 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":189,"threads_started":1}
I20260811 18:54:34.676653 20585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:34.677626 20585 log.cc:824] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:34.678776 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:34.681389 20585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded: No bootstrap required, opened a new log
I20260811 18:54:34.684235 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:34.681995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38515 (local address 127.27.115.254:33091)
0811 18:54:34.682179 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:34.682185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:34.682432 (+ 247us) server_negotiation.cc:407] Connection header received
0811 18:54:34.682536 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:34.682539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:34.682584 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:34.682694 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:34.683636 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:34.683640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:34.683643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:34.683657 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:34.683691 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:34.683705 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:34.683816 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:34.684039 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 18:54:34.684087 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:54:34.686678 20585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } }
I20260811 18:54:34.686949 20585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:34.686982 20585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c251c0b9d440baa37b56a1206deded, State: Initialized, Role: FOLLOWER
I20260811 18:54:34.687392 20585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } }
I20260811 18:54:34.687521 20585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:34.687567 20585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:34.687644 20585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:34.688724 20585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } }
I20260811 18:54:34.689085 20585 leader_election.cc:302] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51c251c0b9d440baa37b56a1206deded; no voters:
I20260811 18:54:34.689404 20585 leader_election.cc:288] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:34.689707 20598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:34.689975 20598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [term 1 LEADER]: Becoming Leader. State: Replica: 51c251c0b9d440baa37b56a1206deded, State: Running, Role: LEADER
I20260811 18:54:34.690413 20598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } }
I20260811 18:54:34.691249 20585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:34.692166 20600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51c251c0b9d440baa37b56a1206deded. Latest consensus state: current_term: 1 leader_uuid: "51c251c0b9d440baa37b56a1206deded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } } }
I20260811 18:54:34.692632 20599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51c251c0b9d440baa37b56a1206deded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c251c0b9d440baa37b56a1206deded" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 33091 } } }
I20260811 18:54:34.692804 20600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:34.692895 20599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:34.693602 20607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:34.697398 20607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:34.704912 20607 catalog_manager.cc:1347] Generated new cluster ID: 19795a71fe0f46c48f3c9a9a02513481
I20260811 18:54:34.705112 20607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:34.724749 20607 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:34.725131 20607 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:34.733151 20607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51c251c0b9d440baa37b56a1206deded: Generated new TSK 0
I20260811 18:54:34.734232 20607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:34.747221 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20512
2026-08-11T18:54:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:34 dist-test-slave-9jjr krb5kdc[20620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:34 dist-test-slave-9jjr krb5kdc[20620](info): set up 2 sockets
Aug 11 18:54:34 dist-test-slave-9jjr krb5kdc[20620](info): commencing operation
krb5kdc: starting...
W20260811 18:54:35.899886 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:35 dist-test-slave-9jjr krb5kdc[20620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:36.057384 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45479
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:36.189028 20635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:36.189459 20635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:36.189567 20635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:36.194244 20635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:36.194396 20635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:36.194432 20635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:36.194478 20635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:36.194519 20635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:36.201481 20635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:36.203075 20635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:36.204398 20635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:36.215430 20643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:36.215430 20641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:36.215440 20640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:36.215958 20635 server_base.cc:1034] running on GCE node
I20260811 18:54:36.216881 20635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:36.218693 20635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:36.220105 20635 hybrid_clock.cc:648] HybridClock initialized: now 1786474476219960 us; error 61 us; skew 500 ppm
Aug 11 18:54:36 dist-test-slave-9jjr krb5kdc[20620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474476, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:36.224063 20635 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:36.225366 20635 webserver.cc:460] Webserver started at http://127.27.115.254:34665/ using document root <none> and password file <none>
I20260811 18:54:36.226531 20635 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:36.226655 20635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:36.226944 20635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:36.229450 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "4c68649dafe04f03bd1a972002481022"
format_stamp: "Formatted at 2026-08-11 18:54:36 on dist-test-slave-9jjr"
server_key: "3cc5b390fd4408fe2daa0ac412bb9da1"
server_key_iv: "84da7a0c0cebf76d1204b8a9450146e5"
server_key_version: "encryptionkey@0"
I20260811 18:54:36.230261 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "4c68649dafe04f03bd1a972002481022"
format_stamp: "Formatted at 2026-08-11 18:54:36 on dist-test-slave-9jjr"
server_key: "3cc5b390fd4408fe2daa0ac412bb9da1"
server_key_iv: "84da7a0c0cebf76d1204b8a9450146e5"
server_key_version: "encryptionkey@0"
I20260811 18:54:36.235054 20635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:54:36.238268 20651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:36.239784 20635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:54:36.240001 20635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "4c68649dafe04f03bd1a972002481022"
format_stamp: "Formatted at 2026-08-11 18:54:36 on dist-test-slave-9jjr"
server_key: "3cc5b390fd4408fe2daa0ac412bb9da1"
server_key_iv: "84da7a0c0cebf76d1204b8a9450146e5"
server_key_version: "encryptionkey@0"
I20260811 18:54:36.240177 20635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:36.255072 20635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:36.259045 20635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:36.259290 20635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:36.267258 20635 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:45479
I20260811 18:54:36.267294 20706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:45479 every 8 connection(s)
I20260811 18:54:36.268350 20635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:36.271641 20707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:36.277451 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20635
I20260811 18:54:36.277645 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:36.277951 28111 external_mini_cluster.cc:1428] Setting key 16ef99bad76e22d4078020ee3891b78b
I20260811 18:54:36.278712 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: Bootstrap starting.
I20260811 18:54:36.281705 20707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:36.282938 20707 log.cc:824] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:36.285564 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: No bootstrap required, opened a new log
Aug 11 18:54:36 dist-test-slave-9jjr krb5kdc[20620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:36.290851 20707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } }
I20260811 18:54:36.291193 20707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:36.291286 20707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c68649dafe04f03bd1a972002481022, State: Initialized, Role: FOLLOWER
I20260811 18:54:36.291884 20707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } }
I20260811 18:54:36.292060 20707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:36.292167 20707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:36.292313 20707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:36.293522 20707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } }
I20260811 18:54:36.293970 20707 leader_election.cc:302] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c68649dafe04f03bd1a972002481022; no voters:
I20260811 18:54:36.294464 20707 leader_election.cc:288] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:36.294555 20712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:36.294852 20712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [term 1 LEADER]: Becoming Leader. State: Replica: 4c68649dafe04f03bd1a972002481022, State: Running, Role: LEADER
I20260811 18:54:36.295403 20712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } }
I20260811 18:54:36.296391 20707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:36.296959 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:36.279756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43497 (local address 127.27.115.254:45479)
0811 18:54:36.280366 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:36.280383 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:36.280426 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:54:36.281209 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:36.281229 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:36.281651 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:36.282057 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:36.283220 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:36.284279 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:36.285476 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:36.286071 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:36.286298 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:36.289603 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:36.289610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:36.289629 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:36.289667 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:36.291950 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:36.292650 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:36.292657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:36.292662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:36.292807 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:36.293142 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:36.293145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:36.293147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:36.293667 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:36.293953 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:36.294316 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 18:54:36.295261 (+ 945us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":190,"threads_started":1}
I20260811 18:54:36.297717 20713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c68649dafe04f03bd1a972002481022. Latest consensus state: current_term: 1 leader_uuid: "4c68649dafe04f03bd1a972002481022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } } }
I20260811 18:54:36.298530 20713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:36.298337 20714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c68649dafe04f03bd1a972002481022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c68649dafe04f03bd1a972002481022" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45479 } } }
I20260811 18:54:36.298970 20714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:36.302238 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 18:54:36.303395 20727 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:36.303519 20727 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:36.303782 20726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:36.304880 20726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:36.312036 20726 catalog_manager.cc:1347] Generated new cluster ID: 7ea011b95a664b4ab9b6971cb330a2e2
I20260811 18:54:36.312131 20726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:36.320477 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:36.307024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38797 (local address 127.27.115.254:45479)
0811 18:54:36.307153 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:36.307158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:36.309515 (+ 2357us) server_negotiation.cc:407] Connection header received
0811 18:54:36.310099 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:36.310103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:36.310190 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:36.310295 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:36.311307 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:36.312204 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:36.313521 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:36.313941 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:36.314052 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:36.316188 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:36.316194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:36.316197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:36.316253 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:36.318465 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:36.319054 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:36.319057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:36.319059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:36.319166 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:36.319498 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:36.319502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:36.319504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:36.319737 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:36.319891 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:36.320172 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:54:36.320305 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260811 18:54:36.322264 20726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:36.323295 20726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:36.333811 20726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c68649dafe04f03bd1a972002481022: Generated new TSK 0
I20260811 18:54:36.334939 20726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:36.345924 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20635
2026-08-11T18:54:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:36 dist-test-slave-9jjr krb5kdc[20742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:36 dist-test-slave-9jjr krb5kdc[20742](info): set up 2 sockets
Aug 11 18:54:36 dist-test-slave-9jjr krb5kdc[20742](info): commencing operation
krb5kdc: starting...
W20260811 18:54:37.491351 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:37 dist-test-slave-9jjr krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:37.647529 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37661
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:37661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:37.794369 20755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:37.794736 20755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:37.794822 20755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:37.799762 20755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:37.799918 20755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:37.799958 20755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:37.800025 20755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:37.800066 20755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:37.806769 20755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:37661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:37661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:37.808393 20755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:37.809679 20755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:37.820041 20760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:37.820166 20764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:37.820161 20761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:37.820629 20755 server_base.cc:1034] running on GCE node
I20260811 18:54:37.821483 20755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:37.822888 20755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:37.824102 20755 hybrid_clock.cc:648] HybridClock initialized: now 1786474477824056 us; error 64 us; skew 500 ppm
Aug 11 18:54:37 dist-test-slave-9jjr krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474477, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:37.827589 20755 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:37.828691 20755 webserver.cc:460] Webserver started at http://127.27.115.254:45233/ using document root <none> and password file <none>
I20260811 18:54:37.829345 20755 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:37.829414 20755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:37.829694 20755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:37.832146 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "c4f3f93d48b04f6c992342b24dbb4729"
format_stamp: "Formatted at 2026-08-11 18:54:37 on dist-test-slave-9jjr"
server_key: "b23a15f485d058a37e04c032f492c043"
server_key_iv: "f7f22e234f6d8501924ca8db9d527574"
server_key_version: "encryptionkey@0"
I20260811 18:54:37.832803 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "c4f3f93d48b04f6c992342b24dbb4729"
format_stamp: "Formatted at 2026-08-11 18:54:37 on dist-test-slave-9jjr"
server_key: "b23a15f485d058a37e04c032f492c043"
server_key_iv: "f7f22e234f6d8501924ca8db9d527574"
server_key_version: "encryptionkey@0"
I20260811 18:54:37.837458 20755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 18:54:37.840390 20772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:37.841838 20755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:54:37.842027 20755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "c4f3f93d48b04f6c992342b24dbb4729"
format_stamp: "Formatted at 2026-08-11 18:54:37 on dist-test-slave-9jjr"
server_key: "b23a15f485d058a37e04c032f492c043"
server_key_iv: "f7f22e234f6d8501924ca8db9d527574"
server_key_version: "encryptionkey@0"
I20260811 18:54:37.842206 20755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:37.859467 20755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:37.863405 20755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:37.863672 20755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:37.872691 20755 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:37661
I20260811 18:54:37.872799 20826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:37661 every 8 connection(s)
I20260811 18:54:37.873929 20755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:37.877287 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:37.878085 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20755
I20260811 18:54:37.878271 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:37.878623 28111 external_mini_cluster.cc:1428] Setting key 98103fdeaffa7289542eea18deb8ea69
I20260811 18:54:37.885224 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: Bootstrap starting.
I20260811 18:54:37.888661 20827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:37.890057 20827 log.cc:824] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:37.890813 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:37.880491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43837 (local address 127.27.115.254:37661)
0811 18:54:37.881204 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:37.881224 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:37.881278 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:54:37.882067 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:37.882089 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:37.882602 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:37.883040 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:37.883570 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:37.884780 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:37.886024 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:37.886762 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:37.887014 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:37.888233 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:37.888240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:37.888260 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:37.888897 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:37.888986 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:37.889020 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:37.889161 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:37.889379 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 18:54:37.889679 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":199,"threads_started":1}
I20260811 18:54:37.893162 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: No bootstrap required, opened a new log
I20260811 18:54:37.894999 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:37.898820 20827 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } }
I20260811 18:54:37.899156 20827 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:37.899255 20827 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f3f93d48b04f6c992342b24dbb4729, State: Initialized, Role: FOLLOWER
I20260811 18:54:37.899873 20827 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } }
I20260811 18:54:37.900068 20827 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:37.900135 20827 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:37.900244 20827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:37.901621 20827 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } }
I20260811 18:54:37.902067 20827 leader_election.cc:302] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4f3f93d48b04f6c992342b24dbb4729; no voters:
I20260811 18:54:37.902550 20827 leader_election.cc:288] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:37.902686 20840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:37.903033 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:37.897720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53597 (local address 127.27.115.254:37661)
0811 18:54:37.897883 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:37.897891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:37.898214 (+ 323us) server_negotiation.cc:407] Connection header received
0811 18:54:37.898296 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:37.898299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:37.898347 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:37.898436 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:37.898801 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:37.899523 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:37.900761 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:37.901042 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:37.901159 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:37.902213 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:37.902263 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:37.902266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:37.902285 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:37.902324 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:37.902338 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:37.902424 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:37.902588 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 18:54:37.902659 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:54:37.903004 20840 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [term 1 LEADER]: Becoming Leader. State: Replica: c4f3f93d48b04f6c992342b24dbb4729, State: Running, Role: LEADER
I20260811 18:54:37.903476 20840 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } }
I20260811 18:54:37.904181 20827 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:37.904860 20840 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4f3f93d48b04f6c992342b24dbb4729. Latest consensus state: current_term: 1 leader_uuid: "c4f3f93d48b04f6c992342b24dbb4729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } } }
I20260811 18:54:37.905067 20840 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:37.905455 20841 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4f3f93d48b04f6c992342b24dbb4729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4f3f93d48b04f6c992342b24dbb4729" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 37661 } } }
I20260811 18:54:37.905613 20841 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729 [sys.catalog]: This master's current role is: LEADER
W20260811 18:54:37.909490 20855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:37.909605 20855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:37.909720 20848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:37.910642 20848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:37.917429 20848 catalog_manager.cc:1347] Generated new cluster ID: 9e148ef4cf69419797ce8f2d2812952f
I20260811 18:54:37.917531 20848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:37.936920 20848 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:37.938103 20848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:37.958006 20848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4f3f93d48b04f6c992342b24dbb4729: Generated new TSK 0
I20260811 18:54:37.959106 20848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:37.971828 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20755
2026-08-11T18:54:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:38 dist-test-slave-9jjr krb5kdc[20862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:38 dist-test-slave-9jjr krb5kdc[20862](info): set up 2 sockets
Aug 11 18:54:38 dist-test-slave-9jjr krb5kdc[20862](info): commencing operation
krb5kdc: starting...
W20260811 18:54:39.143700 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:39 dist-test-slave-9jjr krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:39.298380 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36397
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:39.424547 20875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:39.424923 20875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:39.425017 20875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:39.429563 20875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:39.429709 20875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:39.429747 20875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:39.429786 20875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:39.429826 20875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:39.435804 20875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:36397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:36397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:39.437063 20875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:39.438262 20875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:39.448669 20881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:39.448658 20880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:39.448699 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:39.449190 20875 server_base.cc:1034] running on GCE node
I20260811 18:54:39.450003 20875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:39.451421 20875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:39.452718 20875 hybrid_clock.cc:648] HybridClock initialized: now 1786474479452706 us; error 33 us; skew 500 ppm
Aug 11 18:54:39 dist-test-slave-9jjr krb5kdc[20862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474479, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:39.456004 20875 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:39.457152 20875 webserver.cc:460] Webserver started at http://127.27.115.254:43771/ using document root <none> and password file <none>
I20260811 18:54:39.457767 20875 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:39.457834 20875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:39.458011 20875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:39.460282 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "2ceee95faf5142139ed745424abca552"
format_stamp: "Formatted at 2026-08-11 18:54:39 on dist-test-slave-9jjr"
server_key: "e273dde6b7efc1dfdcc0b4472f4c9453"
server_key_iv: "c7e4c46eaa3e84825214a3827ae5ddee"
server_key_version: "encryptionkey@0"
I20260811 18:54:39.460820 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "2ceee95faf5142139ed745424abca552"
format_stamp: "Formatted at 2026-08-11 18:54:39 on dist-test-slave-9jjr"
server_key: "e273dde6b7efc1dfdcc0b4472f4c9453"
server_key_iv: "c7e4c46eaa3e84825214a3827ae5ddee"
server_key_version: "encryptionkey@0"
I20260811 18:54:39.465309 20875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:54:39.468232 20893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:39.469686 20875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:54:39.469879 20875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "2ceee95faf5142139ed745424abca552"
format_stamp: "Formatted at 2026-08-11 18:54:39 on dist-test-slave-9jjr"
server_key: "e273dde6b7efc1dfdcc0b4472f4c9453"
server_key_iv: "c7e4c46eaa3e84825214a3827ae5ddee"
server_key_version: "encryptionkey@0"
I20260811 18:54:39.470021 20875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:39.475212 20875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:39.478925 20875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:39.479173 20875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:39.487241 20875 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:36397
I20260811 18:54:39.487270 20949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:36397 every 8 connection(s)
I20260811 18:54:39.488371 20875 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:39.491699 20950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:39.497915 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 20875
I20260811 18:54:39.498093 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:39.498364 20950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552: Bootstrap starting.
I20260811 18:54:39.498440 28111 external_mini_cluster.cc:1428] Setting key c859f7cc9dc5ebf5f6ea9e6d0566be79
I20260811 18:54:39.501420 20950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:39.502990 20950 log.cc:824] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:39.505053 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:39.499993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36595 (local address 127.27.115.254:36397)
0811 18:54:39.500518 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:39.500534 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:39.500575 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:54:39.501324 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:39.501343 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:39.501722 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:39.502096 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:39.503285 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:39.503292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:39.503310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:39.503323 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:39.503401 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:39.503441 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:39.503573 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:39.503784 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:54:39.503991 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":166,"threads_started":1}
I20260811 18:54:39.505844 20950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552: No bootstrap required, opened a new log
I20260811 18:54:39.508684 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:39.510831 20950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } }
I20260811 18:54:39.511189 20950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:39.511566 20950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ceee95faf5142139ed745424abca552, State: Initialized, Role: FOLLOWER
I20260811 18:54:39.512260 20950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } }
I20260811 18:54:39.512542 20950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:39.512630 20950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:39.512759 20950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:39.513600 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:39.511347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49995 (local address 127.27.115.254:36397)
0811 18:54:39.511524 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:39.511529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:39.511784 (+ 255us) server_negotiation.cc:407] Connection header received
0811 18:54:39.511994 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:39.511998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:39.512054 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:39.512144 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:39.513119 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:39.513122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:39.513124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:54:39.513138 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:39.513171 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:54:39.513183 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:39.513273 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:39.513395 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 18:54:39.513435 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:54:39.514531 20950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } }
I20260811 18:54:39.515022 20950 leader_election.cc:302] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ceee95faf5142139ed745424abca552; no voters:
I20260811 18:54:39.515460 20950 leader_election.cc:288] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:39.515617 20965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:39.515921 20965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [term 1 LEADER]: Becoming Leader. State: Replica: 2ceee95faf5142139ed745424abca552, State: Running, Role: LEADER
I20260811 18:54:39.516396 20965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } }
I20260811 18:54:39.517082 20950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:39.518407 20967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ceee95faf5142139ed745424abca552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } } }
I20260811 18:54:39.518751 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ceee95faf5142139ed745424abca552. Latest consensus state: current_term: 1 leader_uuid: "2ceee95faf5142139ed745424abca552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ceee95faf5142139ed745424abca552" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 36397 } } }
I20260811 18:54:39.518888 20967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:39.519016 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:39.519598 20977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:39.522873 20977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:39.529295 20977 catalog_manager.cc:1347] Generated new cluster ID: 79e6cc85bc8f43fd808150c610a40177
I20260811 18:54:39.529392 20977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:39.544154 20977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:39.544559 20977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:39.559716 20977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ceee95faf5142139ed745424abca552: Generated new TSK 0
I20260811 18:54:39.560707 20977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:54:39.570792 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 20875
2026-08-11T18:54:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1600 ms)
[----------] 18 tests from AuthTokenIssuingTest (28115 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:39 dist-test-slave-9jjr krb5kdc[20989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:39 dist-test-slave-9jjr krb5kdc[20989](info): set up 2 sockets
Aug 11 18:54:39 dist-test-slave-9jjr krb5kdc[20989](info): commencing operation
krb5kdc: starting...
W20260811 18:54:40.723311 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:40 dist-test-slave-9jjr krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:40.869696 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45617
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:40.992419 21002 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:40.992759 21002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:40.992784 21002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:40.992887 21002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:40.997205 21002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:40.997298 21002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:40.997320 21002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:40.997339 21002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:40.997354 21002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:41.003341 21002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:41.004658 21002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:41.005827 21002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:41.015625 21008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:41.015679 21007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:41.015592 21010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:41.015981 21002 server_base.cc:1034] running on GCE node
I20260811 18:54:41.016867 21002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:41.018328 21002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:41.019562 21002 hybrid_clock.cc:648] HybridClock initialized: now 1786474481019541 us; error 40 us; skew 500 ppm
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:41.023228 21002 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:41.024370 21002 webserver.cc:460] Webserver started at http://127.27.115.254:45853/ using document root <none> and password file <none>
I20260811 18:54:41.025075 21002 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:41.025192 21002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:41.025471 21002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:41.028051 21002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "5251602ac9d24f84a45414a405bfdbc8"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "ea545237d1cd96bd687e428cc8988a94"
server_key_iv: "37b63a0acbbdbcedb55727b85806189d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.028731 21002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "5251602ac9d24f84a45414a405bfdbc8"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "ea545237d1cd96bd687e428cc8988a94"
server_key_iv: "37b63a0acbbdbcedb55727b85806189d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.033697 21002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 18:54:41.036950 21018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:41.038589 21002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:54:41.038790 21002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "5251602ac9d24f84a45414a405bfdbc8"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "ea545237d1cd96bd687e428cc8988a94"
server_key_iv: "37b63a0acbbdbcedb55727b85806189d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.038942 21002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:41.065287 21002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:41.069106 21002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:41.069358 21002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:41.077310 21002 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:45617
I20260811 18:54:41.077632 21077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:45617 every 8 connection(s)
I20260811 18:54:41.078522 21002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:41.079231 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21002
I20260811 18:54:41.079377 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:41.079700 28111 external_mini_cluster.cc:1428] Setting key c07e781dfbe7bc97425468a6e2b2a0be
I20260811 18:54:41.082690 21078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:41.088138 21078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.115.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.253@KRBTEST.COM" created.
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:41.106345 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.090844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.27.115.254:45617)
0811 18:54:41.091361 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.091381 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.091675 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:54:41.093128 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.093132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.093342 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.093541 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.094595 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.095502 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.096555 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.096958 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.097150 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.100333 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.100343 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.100363 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.100401 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.102779 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.103535 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.103541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.103543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.103666 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.103993 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.103995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.103997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.104449 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.104568 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.105104 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.105355 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":357,"thread_start_us":165,"threads_started":1}
I20260811 18:54:41.106351 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.090844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45617 (local address 127.0.0.1:50054)
0811 18:54:41.091478 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.091521 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.092749 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.093539 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.093544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.094066 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.094430 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.094446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.095644 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.095648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.096421 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.096431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.097141 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.097145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.097324 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.098041 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.098059 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.100105 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.102935 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.102947 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.102958 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.103380 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.103761 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.103764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.103765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.103898 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.104568 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.104574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.104783 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.104970 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.105441 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":194,"spinlock_wait_cycles":47872,"thread_start_us":197,"threads_started":1}
I20260811 18:54:41.109263 21078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:41.110222 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.109855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42298)
0811 18:54:41.109991 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.110073 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 18:54:41.110435 21020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
I20260811 18:54:41.111449 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.111233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42306)
0811 18:54:41.111342 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.111392 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:54:41.111615 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33829: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
I20260811 18:54:41.113283 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:33829
--webserver_interface=127.27.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:54:41.160966 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } attempt: 1
I20260811 18:54:41.162120 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.161754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42312)
0811 18:54:41.161959 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.162030 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 18:54:41.163134 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.162871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42316)
0811 18:54:41.162992 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.163044 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:54:41.163314 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33829: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
I20260811 18:54:41.231477 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } attempt: 2
I20260811 18:54:41.232474 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.232087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42318)
0811 18:54:41.232285 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.232372 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 18:54:41.233392 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.233060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42320)
0811 18:54:41.233202 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.233283 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 18:54:41.233587 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:33829: Network error: Client connection negotiation failed: client connection to 127.27.115.253:33829: connect: Connection refused (error 111)
W20260811 18:54:41.236482 21087 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:41.236835 21087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:41.236891 21087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:41.236991 21087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:41.241477 21087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:41.241544 21087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:41.241565 21087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:41.241582 21087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:41.241598 21087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:41.247514 21087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:41.248694 21087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:41.249789 21087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:41.259059 21096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:41.259059 21094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:41.259420 21093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:41.259831 21087 server_base.cc:1034] running on GCE node
I20260811 18:54:41.260524 21087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:41.261758 21087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:41.262957 21087 hybrid_clock.cc:648] HybridClock initialized: now 1786474481262942 us; error 33 us; skew 500 ppm
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:41.266115 21087 init.cc:377] Logged in from keytab as kudu/127.27.115.253@KRBTEST.COM (short username kudu)
I20260811 18:54:41.267238 21087 webserver.cc:460] Webserver started at http://127.27.115.253:44077/ using document root <none> and password file <none>
I20260811 18:54:41.267863 21087 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:41.267978 21087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:41.268275 21087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:41.270617 21087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data/instance:
uuid: "bb1659f4d6444b08a0a024b2389dc2ac"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "604ac2b507b71567de4a8b5368e5a925"
server_key_iv: "f3bec39eb63eb8c6616b987ef10d740d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.271307 21087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal/instance:
uuid: "bb1659f4d6444b08a0a024b2389dc2ac"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "604ac2b507b71567de4a8b5368e5a925"
server_key_iv: "f3bec39eb63eb8c6616b987ef10d740d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.275664 21087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:41.278486 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:41.279852 21087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:41.280023 21087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "bb1659f4d6444b08a0a024b2389dc2ac"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "604ac2b507b71567de4a8b5368e5a925"
server_key_iv: "f3bec39eb63eb8c6616b987ef10d740d"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.280169 21087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:41.312716 21087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:41.316404 21087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:41.316638 21087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:41.324766 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } attempt: 3
I20260811 18:54:41.325325 21087 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.253:33829
I20260811 18:54:41.325651 21163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.253:33829 every 8 connection(s)
I20260811 18:54:41.326766 21087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
I20260811 18:54:41.331164 21165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:41.332204 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21087
I20260811 18:54:41.332345 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-1/wal/instance
I20260811 18:54:41.332710 28111 external_mini_cluster.cc:1428] Setting key 4a60e89f2d9d3f4df460a17942cf830f
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:41.336838 21165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:41.342583 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.325388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42322)
0811 18:54:41.325610 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.325645 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.325768 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.329445 (+ 3677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.329449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.329673 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.329843 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.329850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.332270 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.332277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.333209 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.333233 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.334400 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.334408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.334569 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.335262 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.335283 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.337048 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.340284 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.340287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.340289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.340586 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.341097 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.341103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.341107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.341231 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.341927 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.341931 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.342067 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.342231 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.342399 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
WARNING: no policy specified for kudu/127.27.115.252@KRBTEST.COM; defaulting to no policy
I20260811 18:54:41.344367 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.326789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.27.115.253:33829)
0811 18:54:41.327411 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.327434 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.327485 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:54:41.328436 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.328461 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.328976 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.329439 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.330014 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.332050 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.333405 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.334211 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.334399 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.337325 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.337335 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.337358 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.337404 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.340132 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.340746 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.340755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.340765 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.340952 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.341320 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.341324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.341325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.341786 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.341951 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.342414 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.343208 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":364,"server-negotiator.queue_time_us":394,"thread_start_us":177,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.252@KRBTEST.COM" created.
I20260811 18:54:41.347121 21078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:41.347949 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.347676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57586)
0811 18:54:41.347815 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.347874 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:54:41.348821 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.348514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57600)
0811 18:54:41.348686 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.348742 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 18:54:41.349001 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.352097 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.339324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.27.115.254:45617)
0811 18:54:41.339476 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.339481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.339886 (+ 405us) server_negotiation.cc:407] Connection header received
0811 18:54:41.340141 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.340150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.340197 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.340277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.341314 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.342258 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.344199 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.345194 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.345310 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.348193 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.348197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.348201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.348228 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.349956 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.350684 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.350687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.350689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.350784 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.351136 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.351139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.351140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.351335 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.351416 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.351873 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.351981 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:41.352376 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.339324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45617 (local address 127.0.0.1:50064)
0811 18:54:41.339748 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.339801 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.340009 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.340483 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.340491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.340934 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.341156 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.341168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.342459 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.342463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.344014 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.344027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.345331 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.345334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.345432 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.346100 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.346115 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.347996 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.350096 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.350113 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.350121 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.350524 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.350909 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.350915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.350920 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.351030 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.351453 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.351463 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.351632 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.351804 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.352046 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260811 18:54:41.353688 21165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:41.364111 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.354341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42338)
0811 18:54:41.354444 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.354455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.354525 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.354884 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.354886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.355029 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.355131 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.355136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.356227 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.356230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.356762 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.356771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.357427 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.357430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.357509 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.358064 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.358100 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.360559 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.362510 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.362513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.362515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.362720 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.363007 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.363012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.363014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.363113 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.363518 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.363521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.363618 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.363729 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.363912 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:54:41.364111 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.354510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.27.115.253:33829)
0811 18:54:41.354641 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.354648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.354665 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:54:41.354708 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.354711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.354764 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.354855 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.355262 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.356092 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.356900 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.357292 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.357406 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.360722 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.360726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.360728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.360755 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.362354 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.362827 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.362832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.362834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.362914 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.363210 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.363215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.363217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.363410 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.363530 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.363778 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.363978 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
I20260811 18:54:41.365336 21165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:41.366015 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:40139
--webserver_interface=127.27.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:54:41.366305 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.365864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57604)
0811 18:54:41.366018 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.366209 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 18:54:41.366943 21108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.367975 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.367674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57614)
0811 18:54:41.367831 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.367901 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 18:54:41.368289 21165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.380199 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 1
I20260811 18:54:41.381253 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.380890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57620)
0811 18:54:41.381098 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.381167 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 18:54:41.382233 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.381894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57636)
0811 18:54:41.382009 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.382070 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 18:54:41.382375 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.417637 21165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 1
I20260811 18:54:41.418629 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.418327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57648)
0811 18:54:41.418483 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.418548 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 18:54:41.419544 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.419210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57658)
0811 18:54:41.419433 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.419489 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260811 18:54:41.419679 21165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.457552 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 2
I20260811 18:54:41.458518 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.458206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57666)
0811 18:54:41.458369 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.458431 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 18:54:41.459270 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.459039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57676)
0811 18:54:41.459169 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.459221 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 18:54:41.459407 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.487872 21165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 2
I20260811 18:54:41.488893 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.488524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57680)
0811 18:54:41.488725 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.488800 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 18:54:41.489326 21172 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:41.489719 21172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260811 18:54:41.489748 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.489481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57684)
0811 18:54:41.489608 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.489661 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:54:41.489782 21172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:41.489866 21172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:41.489892 21165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
W20260811 18:54:41.494107 21172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:41.494241 21172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:41.494264 21172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:41.494304 21172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:41.494339 21172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:41.499877 21172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:45617,127.27.115.253:33829,127.27.115.252:40139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:41.501025 21172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:41.502099 21172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:41.510805 21181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:41.510914 21172 server_base.cc:1034] running on GCE node
W20260811 18:54:41.510838 21178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:41.510807 21179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:41.511672 21172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:41.512835 21172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:41.514070 21172 hybrid_clock.cc:648] HybridClock initialized: now 1786474481514042 us; error 40 us; skew 500 ppm
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:41.517069 21172 init.cc:377] Logged in from keytab as kudu/127.27.115.252@KRBTEST.COM (short username kudu)
I20260811 18:54:41.517954 21172 webserver.cc:460] Webserver started at http://127.27.115.252:42215/ using document root <none> and password file <none>
I20260811 18:54:41.518554 21172 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:41.518628 21172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:41.518800 21172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:41.520864 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data/instance:
uuid: "7b08431c17804817a00b83e909c18320"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "2fe705a8d36e0475e6f5739afa45ed52"
server_key_iv: "7417d2460f12f0b1034f27d54f2b1af7"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.521379 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal/instance:
uuid: "7b08431c17804817a00b83e909c18320"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "2fe705a8d36e0475e6f5739afa45ed52"
server_key_iv: "7417d2460f12f0b1034f27d54f2b1af7"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.525292 21172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:54:41.527894 21188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:41.529181 21172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:54:41.529325 21172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "7b08431c17804817a00b83e909c18320"
format_stamp: "Formatted at 2026-08-11 18:54:41 on dist-test-slave-9jjr"
server_key: "2fe705a8d36e0475e6f5739afa45ed52"
server_key_iv: "7417d2460f12f0b1034f27d54f2b1af7"
server_key_version: "encryptionkey@0"
I20260811 18:54:41.529455 21172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:41.558604 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 3
I20260811 18:54:41.559621 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.559288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57694)
0811 18:54:41.559452 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.559533 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 18:54:41.560449 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.560207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57706)
0811 18:54:41.560328 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.560388 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:54:41.560624 21078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.581111 21165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 3
I20260811 18:54:41.581389 21172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:41.582049 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.581738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57716)
0811 18:54:41.581883 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.581964 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:54:41.583192 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.582829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57728)
0811 18:54:41.583036 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.583112 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 18:54:41.583582 21165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:40139: Network error: Client connection negotiation failed: client connection to 127.27.115.252:40139: connect: Connection refused (error 111)
I20260811 18:54:41.585180 21172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:41.585388 21172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:41.593065 21172 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.252:40139
I20260811 18:54:41.593369 21251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.252:40139 every 8 connection(s)
I20260811 18:54:41.594213 21172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
I20260811 18:54:41.594952 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21172
I20260811 18:54:41.595106 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474173680671-28111-0/minicluster-data/master-2/wal/instance
I20260811 18:54:41.595417 28111 external_mini_cluster.cc:1428] Setting key 05cd2f82f9442e5fccdf59b0d06fc778
I20260811 18:54:41.597474 21253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:41.602599 21253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:41.612516 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.596822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.27.115.252:40139)
0811 18:54:41.597260 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.597282 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.597510 (+ 228us) server_negotiation.cc:407] Connection header received
0811 18:54:41.598550 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.598573 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.599012 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.599421 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.600118 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.601702 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.602699 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.603438 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.603629 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.606625 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.606631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.606643 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.606676 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.608705 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.609273 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.609281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.609288 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.609454 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.609800 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.609804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.609805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.610382 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.610602 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.610968 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.611359 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
I20260811 18:54:41.616000 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.605050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.27.115.254:45617)
0811 18:54:41.605234 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.605241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.605598 (+ 357us) server_negotiation.cc:407] Connection header received
0811 18:54:41.605736 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.605740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.605795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.605890 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.606769 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.607444 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.608736 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.609128 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.609271 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.611850 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.611854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.611857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.611913 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.613680 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.614344 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.614348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.614349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.614449 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.614779 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.614783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.614787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.615102 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.615208 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.615687 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.615855 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:54:41.616557 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.604951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:45617 (local address 127.0.0.1:50068)
0811 18:54:41.605430 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.605490 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.605626 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.605970 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.605978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.606446 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.606638 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.606645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.607567 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.607570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.608586 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.608602 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.609321 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.609326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.609465 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.610094 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.610107 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.611664 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.613832 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.613843 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.613847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.614187 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.614556 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.614560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.614562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.614669 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.615231 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.615237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.615375 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.616182 (+ 807us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.616385 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":198,"threads_started":1}
I20260811 18:54:41.617771 21253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:41.626370 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.615591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.27.115.254:45617)
0811 18:54:41.615920 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.615935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.615946 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:54:41.615977 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.615978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.616015 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.616119 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.617018 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.617698 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.618779 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.619036 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.619150 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.622059 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.622063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.622065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.622094 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.624083 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.624716 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.624721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.624724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.624853 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.625240 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.625244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.625247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.625566 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.625835 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.625955 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.626094 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
I20260811 18:54:41.628367 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.618364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:33829 (local address 127.0.0.1:42350)
0811 18:54:41.618647 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.618659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.618727 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.618998 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.619001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.619196 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.619299 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.619304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.620180 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.620186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.620746 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.620757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.621226 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.621230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.621349 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.621934 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.621947 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.623264 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.626535 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.626541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.626545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.626826 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.627208 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.627212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.627214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.627330 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.627786 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.627790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.627904 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.628026 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.628199 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260811 18:54:41.628379 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.618515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.27.115.253:33829)
0811 18:54:41.618670 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.618675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.618696 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:54:41.618842 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.618845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.618889 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.618977 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.619418 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.620062 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.620873 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.621095 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.621212 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.623453 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.623459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.623462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.623498 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.626388 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.626934 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.626939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.626941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.627043 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.627434 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.627437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.627439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.627675 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.627795 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.628087 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.628205 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:54:41.629739 21253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
I20260811 18:54:41.638572 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.628056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.27.115.253:33829)
0811 18:54:41.628493 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.628505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.628523 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:41.628582 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.628587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.628645 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.628733 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.629616 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.630623 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.631579 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.631840 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.631922 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.634779 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.634783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.634785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.634817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.636624 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.637112 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.637115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.637117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.637210 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.637510 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.637514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.637516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.637733 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.637956 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.638215 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.638356 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":164,"threads_started":1}
I20260811 18:54:41.640379 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.630294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57758)
0811 18:54:41.630439 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.630452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.630556 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.630921 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.630924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.631088 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.631205 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.631210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.633585 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.633588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.634284 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.634296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.634791 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.634794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.634864 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.635300 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.635312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.636522 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.638390 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.638393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.638395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.638678 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.639162 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.639165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.639167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.639263 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.639771 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.639774 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.639890 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.640005 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.640186 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 18:54:41.640415 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.630471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.27.115.252:40139)
0811 18:54:41.630691 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.630695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.630707 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:54:41.630737 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.630739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.630778 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.630862 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.631329 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.632465 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.634409 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.634673 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.634789 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.636675 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.636678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.636680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.636701 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.638265 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.638792 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.638794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.638796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.639012 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.639370 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.639373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.639375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.639654 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.639785 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.640064 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.640269 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":120}
I20260811 18:54:41.643592 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:41.644609 21253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: Bootstrap starting.
I20260811 18:54:41.645195 21050 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5251602ac9d24f84a45414a405bfdbc8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50078
I20260811 18:54:41.648213 21253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:41.649310 21253 log.cc:824] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:41.652446 21253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: No bootstrap required, opened a new log
I20260811 18:54:41.657099 21253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.657397 21253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b08431c17804817a00b83e909c18320, State: Initialized, Role: FOLLOWER
I20260811 18:54:41.657761 21253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.658397 21261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.658562 21261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.659387 21253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:54:41.665341 21274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:41.665462 21274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:41.664130 21172 master_runner.cc:186] Error getting master registration for 127.27.115.254:45617: OK, instance_id { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" instance_seqno: 1786474481075079 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:54:41.667526 21172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:54:41.724833 21078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 4
I20260811 18:54:41.726755 21165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } attempt: 4
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:54:41 dist-test-slave-9jjr krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474481, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
I20260811 18:54:41.736521 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.725607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57774)
0811 18:54:41.725906 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.725930 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.726075 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.726416 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.726419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.726651 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.726785 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.726793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.727843 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.727847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.728742 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.728756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.729418 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.729422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.729521 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.730181 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.730198 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.731952 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.734417 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.734421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.734423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.734729 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.735075 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.735081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.735085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.735206 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.735838 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.735842 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.735983 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.736119 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.736311 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 18:54:41.736546 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.725936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.27.115.252:40139)
0811 18:54:41.726090 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.726095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.726114 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:54:41.726226 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.726229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.726286 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.726390 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.726931 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.727685 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.728900 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.729269 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.729382 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.732127 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.732131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.732134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.732161 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.734225 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.734857 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.734861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.734863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.734957 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.735342 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.735346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.735348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.735689 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.735802 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.736207 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.736370 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:54:41.740289 21078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: Bootstrap starting.
I20260811 18:54:41.743096 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.727324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:40139 (local address 127.0.0.1:57784)
0811 18:54:41.727462 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:41.727480 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:41.727585 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:41.728445 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:41.728449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:41.728672 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:41.728768 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.728774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.730463 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.730467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.731162 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:41.731184 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.731853 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.731858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:41.731958 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.732586 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:41.732607 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:41.734187 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:41.736595 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.736601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.736603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.736922 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.737298 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:41.737301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:41.737303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:41.737409 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:41.742303 (+ 4894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:41.742307 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:41.742476 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:54:41.742644 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 18:54:41.742842 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 18:54:41.743219 21078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:41.743394 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.727646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.27.115.252:40139)
0811 18:54:41.728144 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.728163 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.728181 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:54:41.728224 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.728227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.728293 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.728407 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.729495 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.730342 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.731335 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.731695 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.731816 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.734364 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.734368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.734371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.734400 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.736435 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.737064 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.737067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.737068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.737188 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.737526 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.737530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.737533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.737788 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.737886 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.742816 (+ 4930us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.743076 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":164,"threads_started":1}
I20260811 18:54:41.744303 21078 log.cc:824] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:41.746767 21078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: No bootstrap required, opened a new log
I20260811 18:54:41.747298 21165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: Bootstrap starting.
I20260811 18:54:41.750151 21078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.750334 21165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:41.750435 21078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5251602ac9d24f84a45414a405bfdbc8, State: Initialized, Role: FOLLOWER
I20260811 18:54:41.750821 21078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.751286 21165 log.cc:824] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:41.752410 21078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:41.752533 21050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:54:41.752674 21050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:41.754700 21050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.756589 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.756726 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.757201 21219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5251602ac9d24f84a45414a405bfdbc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7b08431c17804817a00b83e909c18320"
I20260811 18:54:41.757484 21219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:41.759053 21219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5251602ac9d24f84a45414a405bfdbc8 in term 1.
I20260811 18:54:41.759593 21020 leader_election.cc:302] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5251602ac9d24f84a45414a405bfdbc8, 7b08431c17804817a00b83e909c18320; no voters:
I20260811 18:54:41.760010 21278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:41.760269 21278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [term 1 LEADER]: Becoming Leader. State: Replica: 5251602ac9d24f84a45414a405bfdbc8, State: Running, Role: LEADER
W20260811 18:54:41.760397 21291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:41.760521 21291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:41.760672 21278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.758713 21002 master_runner.cc:186] Error getting master registration for 127.27.115.253:33829: OK, instance_id { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" instance_seqno: 1786474481322601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:54:41.762073 21002 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:54:41.762964 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5251602ac9d24f84a45414a405bfdbc8. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.763175 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:41.763464 21050 leader_election.cc:288] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [CANDIDATE]: Term 1 election: Requested vote from peers bb1659f4d6444b08a0a024b2389dc2ac (127.27.115.253:33829), 7b08431c17804817a00b83e909c18320 (127.27.115.252:40139)
I20260811 18:54:41.766309 21165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: No bootstrap required, opened a new log
I20260811 18:54:41.770661 21295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:41.771896 21295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:41.773267 21165 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.773803 21165 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1659f4d6444b08a0a024b2389dc2ac, State: Initialized, Role: FOLLOWER
I20260811 18:54:41.774498 21165 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } }
I20260811 18:54:41.777618 21136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5251602ac9d24f84a45414a405bfdbc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb1659f4d6444b08a0a024b2389dc2ac"
I20260811 18:54:41.777957 21136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:41.779628 21136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5251602ac9d24f84a45414a405bfdbc8 in term 1.
I20260811 18:54:41.780095 21165 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:41.782207 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.767562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.27.115.254:45617)
0811 18:54:41.767736 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.767741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.769773 (+ 2032us) server_negotiation.cc:407] Connection header received
0811 18:54:41.770005 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.770009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.770065 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.770161 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.771458 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.772217 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.773596 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.774046 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.774359 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.776795 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.776802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.776805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.776852 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.779327 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.780324 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.780331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.780333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.780457 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.780849 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.780853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.780856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.781172 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.781307 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.781772 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.781946 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:54:41.798170 21219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [term 1 FOLLOWER]: Refusing update from remote peer 5251602ac9d24f84a45414a405bfdbc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:54:41.799431 21316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:41.799548 21316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:41.799888 21293 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:41.802450 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.802650 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.804378 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.768295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.27.115.253:33829)
0811 18:54:41.768425 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.768429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.782779 (+ 14350us) server_negotiation.cc:407] Connection header received
0811 18:54:41.782849 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.782855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.782934 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.783035 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.786461 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.787313 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.788508 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.788931 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.789021 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.791259 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.791263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.791266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.791296 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.793396 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.797158 (+ 3762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.797164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.797166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.797343 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.797812 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.797834 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.797836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.803231 (+ 5395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.803426 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.803977 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.804156 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:54:41.807603 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:41.769028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.27.115.252:40139)
0811 18:54:41.769180 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.769184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.785405 (+ 16221us) server_negotiation.cc:407] Connection header received
0811 18:54:41.787902 (+ 2497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.787906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.787972 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.788027 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.789090 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.789902 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.791232 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:41.791673 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:41.791776 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:41.801622 (+ 9846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:41.801644 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:41.801684 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:41.801730 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:41.804392 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.805103 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.805108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.805110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.805304 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:41.805902 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:41.805906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:41.805908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:41.806288 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:41.806519 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:41.807122 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 18:54:41.807349 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:54:41.808285 21261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5251602ac9d24f84a45414a405bfdbc8. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.808496 21261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.809834 21136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [term 1 FOLLOWER]: Refusing update from remote peer 5251602ac9d24f84a45414a405bfdbc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:41.810643 21293 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:41.817126 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5251602ac9d24f84a45414a405bfdbc8. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.817302 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:54:41.819576 21112 tablet.cc:2369] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:54:41.821425 21024 tablet.cc:2369] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:54:41.822943 21293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.823161 21293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:41.825094 21261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.825276 21261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.825239 21310 mvcc.cc:205] Tried to move back new op lower bound from 7317399477431803904 to 7317399477295734784. Current Snapshot: MvccSnapshot[applied={T|T < 7317399477431803904}]
I20260811 18:54:41.827673 21318 mvcc.cc:205] Tried to move back new op lower bound from 7317399477431803904 to 7317399477295734784. Current Snapshot: MvccSnapshot[applied={T|T < 7317399477431803904}]
I20260811 18:54:41.830808 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.830994 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:41.832928 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5251602ac9d24f84a45414a405bfdbc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5251602ac9d24f84a45414a405bfdbc8" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 45617 } } peers { permanent_uuid: "bb1659f4d6444b08a0a024b2389dc2ac" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 33829 } } peers { permanent_uuid: "7b08431c17804817a00b83e909c18320" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 40139 } } }
I20260811 18:54:41.833549 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:41.836249 21295 catalog_manager.cc:1347] Generated new cluster ID: bb2add0ab7cf4714afa39d3c8a7f6942
I20260811 18:54:41.836359 21295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:41.877053 21295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:41.878656 21295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:41.896422 21295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5251602ac9d24f84a45414a405bfdbc8: Generated new TSK 0
I20260811 18:54:41.903685 21295 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:54:41.941567 21258 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.939670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.27.115.254:45617)
0811 18:54:41.939835 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.939840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.940232 (+ 392us) server_negotiation.cc:407] Connection header received
0811 18:54:41.940468 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.940473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.940531 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.940590 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.941402 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50090: BlockingRecv error: recv got EOF from 127.0.0.1:50090 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 18:54:41.942957 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.940238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.27.115.253:33829)
0811 18:54:41.940379 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.940383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.941691 (+ 1308us) server_negotiation.cc:407] Connection header received
0811 18:54:41.941890 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.941893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.941947 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.942007 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.942814 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42376: BlockingRecv error: recv got EOF from 127.0.0.1:42376 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 18:54:41.943221 21276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.941020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.27.115.252:40139)
0811 18:54:41.941174 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.941179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.942067 (+ 888us) server_negotiation.cc:407] Connection header received
0811 18:54:41.942292 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.942296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.942352 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.942406 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.943085 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57798: BlockingRecv error: recv got EOF from 127.0.0.1:57798 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 18:54:41.947547 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.946071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.27.115.253:33829)
0811 18:54:41.946249 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.946253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.946493 (+ 240us) server_negotiation.cc:407] Connection header received
0811 18:54:41.946691 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.946695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.946787 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.946890 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.947424 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42390: BlockingRecv error: recv got EOF from 127.0.0.1:42390 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 18:54:41.952373 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.951293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.27.115.253:33829)
0811 18:54:41.951453 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.951457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.951473 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:54:41.951531 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.951534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.951589 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.951676 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.952248 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42392: BlockingRecv error: recv got EOF from 127.0.0.1:42392 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:54:41.959445 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.957708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.27.115.253:33829)
0811 18:54:41.957863 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.957868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.958317 (+ 449us) server_negotiation.cc:407] Connection header received
0811 18:54:41.958486 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.958489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.958558 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.958661 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.959308 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42408: BlockingRecv error: recv got EOF from 127.0.0.1:42408 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 18:54:41.967895 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.966475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.27.115.253:33829)
0811 18:54:41.966624 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.966629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.966861 (+ 232us) server_negotiation.cc:407] Connection header received
0811 18:54:41.967014 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.967018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.967096 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.967203 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.967755 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42414: BlockingRecv error: recv got EOF from 127.0.0.1:42414 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:54:41.980528 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.978782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.27.115.253:33829)
0811 18:54:41.978935 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.978940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.979560 (+ 620us) server_negotiation.cc:407] Connection header received
0811 18:54:41.979704 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:41.979707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:41.979772 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:41.979871 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:41.980390 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42424: BlockingRecv error: recv got EOF from 127.0.0.1:42424 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 18:54:42.000965 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:41.999468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.27.115.253:33829)
0811 18:54:41.999621 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:41.999627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:41.999924 (+ 297us) server_negotiation.cc:407] Connection header received
0811 18:54:42.000092 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:42.000095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:42.000158 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:42.000264 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:42.000835 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42434: BlockingRecv error: recv got EOF from 127.0.0.1:42434 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:54:42.036657 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:42.035413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.27.115.253:33829)
0811 18:54:42.035558 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:42.035563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:42.035692 (+ 129us) server_negotiation.cc:407] Connection header received
0811 18:54:42.035877 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:42.035881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:42.035945 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:42.036048 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:42.036533 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42448: BlockingRecv error: recv got EOF from 127.0.0.1:42448 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 18:54:42.104388 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:42.103091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.27.115.253:33829)
0811 18:54:42.103284 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:42.103288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:42.103419 (+ 131us) server_negotiation.cc:407] Connection header received
0811 18:54:42.103625 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:42.103628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:42.103680 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:42.103775 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:42.104276 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42450: BlockingRecv error: recv got EOF from 127.0.0.1:42450 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 18:54:42.236469 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:42.234986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.27.115.253:33829)
0811 18:54:42.235180 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:42.235185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:42.235332 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:54:42.235638 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:42.235648 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:42.235704 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:42.235801 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:42.236347 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42452: BlockingRecv error: recv got EOF from 127.0.0.1:42452 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 18:54:42.496351 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:42.494975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.27.115.253:33829)
0811 18:54:42.495120 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:42.495125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:42.495336 (+ 211us) server_negotiation.cc:407] Connection header received
0811 18:54:42.495498 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:42.495501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:42.495562 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:42.495641 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:42.496201 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42458: BlockingRecv error: recv got EOF from 127.0.0.1:42458 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:42.668501 21274 catalog_manager.cc:1259] Loaded cluster ID: bb2add0ab7cf4714afa39d3c8a7f6942
I20260811 18:54:42.668635 21274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: loading cluster ID for follower catalog manager: success
I20260811 18:54:42.670819 21274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: acquiring CA information for follower catalog manager: success
I20260811 18:54:42.672111 21274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b08431c17804817a00b83e909c18320: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:54:42.801102 21316 catalog_manager.cc:1259] Loaded cluster ID: bb2add0ab7cf4714afa39d3c8a7f6942
I20260811 18:54:42.801254 21316 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: loading cluster ID for follower catalog manager: success
I20260811 18:54:42.803145 21316 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: acquiring CA information for follower catalog manager: success
I20260811 18:54:42.804665 21316 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb1659f4d6444b08a0a024b2389dc2ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:54:43.015995 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:43.011260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.27.115.253:33829)
0811 18:54:43.011566 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:43.011570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:43.011610 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:54:43.011784 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:43.011787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:43.011939 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:43.012073 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:43.012456 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.013053 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.014044 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.014323 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.014474 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:43.014941 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:43.015463 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:43.015578 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:43.015756 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:54:43.015826 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":116,"threads_started":1}
I20260811 18:54:43.024252 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:43.020191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.27.115.252:40139)
0811 18:54:43.020457 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:43.020461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:43.020499 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:54:43.020592 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:43.020595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:43.020728 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:43.020833 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:43.021121 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.021691 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.022643 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.022883 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.022985 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:43.023293 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:43.023767 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:43.023867 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:43.023944 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:54:43.024083 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260811 18:54:43.032469 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:43.028473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.27.115.253:33829)
0811 18:54:43.028643 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:43.028648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:43.028809 (+ 161us) server_negotiation.cc:407] Connection header received
0811 18:54:43.028972 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:43.028976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:43.029037 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:43.029119 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:43.029410 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.030204 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.031097 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.031435 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.031555 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:43.031846 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:43.031941 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:43.032018 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:43.032238 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 18:54:43.032315 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:54:43.033119 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:43.027926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.27.115.254:45617)
0811 18:54:43.028387 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:43.028393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:43.028415 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:54:43.028544 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:43.028547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:43.028749 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:43.028907 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:43.029204 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.030078 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.031328 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.031698 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.031839 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:43.032223 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:43.032615 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:43.032701 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:43.032881 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 18:54:43.032949 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
I20260811 18:54:43.033933 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:43.028871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.27.115.252:40139)
0811 18:54:43.029000 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:43.029004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:43.029508 (+ 504us) server_negotiation.cc:407] Connection header received
0811 18:54:43.029675 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:43.029678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:43.029733 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:43.029817 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:43.030073 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.030906 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.032197 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:43.032502 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:43.032628 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:43.033108 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:43.033205 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:43.033256 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:43.033720 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 18:54:43.033785 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:54:43.043272 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21002
I20260811 18:54:43.053270 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21087
I20260811 18:54:43.063683 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21172
2026-08-11T18:54:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3493 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:43 dist-test-slave-9jjr krb5kdc[21414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:43 dist-test-slave-9jjr krb5kdc[21414](info): set up 2 sockets
Aug 11 18:54:43 dist-test-slave-9jjr krb5kdc[21414](info): commencing operation
krb5kdc: starting...
W20260811 18:54:44.206380 28111 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/test-user.keytab.
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
Entry for principal kudu/127.27.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab.
I20260811 18:54:44.351016 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42581
--webserver_interface=127.27.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:44.471262 21428 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:44.471606 21428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:44.471680 21428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:44.471760 21428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:44.475770 21428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:44.475867 21428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:44.475907 21428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:44.475944 21428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:44.475977 21428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:44.481475 21428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.254.keytab
--principal=kudu/127.27.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.254:42581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:44.482712 21428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:44.483803 21428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:44.493145 21434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.493170 21433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.493170 21436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:44.493481 21428 server_base.cc:1034] running on GCE node
I20260811 18:54:44.494263 21428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:44.495488 21428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:44.496680 21428 hybrid_clock.cc:648] HybridClock initialized: now 1786474484496670 us; error 32 us; skew 500 ppm
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:44.499822 21428 init.cc:377] Logged in from keytab as kudu/127.27.115.254@KRBTEST.COM (short username kudu)
I20260811 18:54:44.501065 21428 webserver.cc:460] Webserver started at http://127.27.115.254:41501/ using document root <none> and password file <none>
I20260811 18:54:44.501727 21428 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:44.501834 21428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:44.502096 21428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:44.504487 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data/instance:
uuid: "302bfdb596ec4002b332cc921a2ccb76"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "f122b84432757ea48a99fd9b00d863fe"
server_key_iv: "5a8fc47fc1ecfa77e398e15ad445c4ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.505162 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance:
uuid: "302bfdb596ec4002b332cc921a2ccb76"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "f122b84432757ea48a99fd9b00d863fe"
server_key_iv: "5a8fc47fc1ecfa77e398e15ad445c4ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.509425 21428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 18:54:44.512063 21443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:44.513546 21428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:54:44.513715 21428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal
uuid: "302bfdb596ec4002b332cc921a2ccb76"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "f122b84432757ea48a99fd9b00d863fe"
server_key_iv: "5a8fc47fc1ecfa77e398e15ad445c4ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.513855 21428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:44.525799 21428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:44.529280 21428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:44.529506 21428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:44.537189 21428 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.254:42581
I20260811 18:54:44.537503 21500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.254:42581 every 8 connection(s)
I20260811 18:54:44.538316 21428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:44.539348 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21428
I20260811 18:54:44.539497 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-0/wal/instance
I20260811 18:54:44.539810 28111 external_mini_cluster.cc:1428] Setting key db08926e185f548ea0b3d7b12af249d4
I20260811 18:54:44.541522 21501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:44.545977 21501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.115.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.253@KRBTEST.COM" created.
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:44.564901 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.549087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.27.115.254:42581)
0811 18:54:44.549667 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:44.549690 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:44.550207 (+ 517us) server_negotiation.cc:407] Connection header received
0811 18:54:44.551350 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:44.551359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:44.551725 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:44.551956 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:44.552951 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.554503 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.555550 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.555945 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.556134 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.559240 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:44.559250 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:44.559267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:44.559295 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:44.561404 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.561930 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.561936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.561937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.562051 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.562548 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.562550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.562552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.563125 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:44.563261 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:44.563691 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 18:54:44.563959 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":381,"spinlock_wait_cycles":896,"thread_start_us":191,"threads_started":1}
I20260811 18:54:44.564908 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.549087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42581 (local address 127.0.0.1:59722)
0811 18:54:44.549908 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.549969 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:44.550973 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:44.552018 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:44.552027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:44.552460 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:44.552760 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.552775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.554644 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.554649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.555417 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.555426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.556116 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.556120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.556300 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.557051 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:44.557069 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:44.559004 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:44.561543 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.561553 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.561565 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.561812 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.562249 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.562252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.562254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.562437 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.563288 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:44.563293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:44.563430 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 18:54:44.563581 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 18:54:44.564036 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":171,"spinlock_wait_cycles":44032,"thread_start_us":209,"threads_started":1}
I20260811 18:54:44.567816 21501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:44.568783 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.568427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40202)
0811 18:54:44.568566 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.568670 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 18:54:44.569057 21445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
Entry for principal kudu/127.27.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab.
I20260811 18:54:44.570199 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.569866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40218)
0811 18:54:44.569986 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.570043 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:54:44.570407 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:41459: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
I20260811 18:54:44.570807 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:41459
--webserver_interface=127.27.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:54:44.619683 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } attempt: 1
I20260811 18:54:44.620756 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.620355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40232)
0811 18:54:44.620565 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.620651 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 18:54:44.621922 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.621575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40244)
0811 18:54:44.621760 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.621830 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 18:54:44.622244 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:41459: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
I20260811 18:54:44.690469 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } attempt: 2
I20260811 18:54:44.691504 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.691164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40252)
0811 18:54:44.691347 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.691418 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 18:54:44.692389 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.692128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40260)
0811 18:54:44.692257 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.692317 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:54:44.692525 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:41459: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
W20260811 18:54:44.704973 21509 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:44.705381 21509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:44.705446 21509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:44.705528 21509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:44.710647 21509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:44.710745 21509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:44.710805 21509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:44.710839 21509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:44.710880 21509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:44.717254 21509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.253.keytab
--principal=kudu/127.27.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.253:41459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:44.718734 21509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:44.719933 21509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:44.730077 21518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.730062 21514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.730084 21515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:44.730295 21509 server_base.cc:1034] running on GCE node
I20260811 18:54:44.731249 21509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:44.732767 21509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:44.733997 21509 hybrid_clock.cc:648] HybridClock initialized: now 1786474484733961 us; error 66 us; skew 500 ppm
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:44.737602 21509 init.cc:377] Logged in from keytab as kudu/127.27.115.253@KRBTEST.COM (short username kudu)
I20260811 18:54:44.738896 21509 webserver.cc:460] Webserver started at http://127.27.115.253:44827/ using document root <none> and password file <none>
I20260811 18:54:44.739529 21509 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:44.739604 21509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:44.739830 21509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:44.742785 21509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data/instance:
uuid: "03953defb747457f8fb6b42feefb1a54"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "976f1717a15657a17307884fb528981c"
server_key_iv: "b3b2d7d9781b4d7b528ae5d44171b3a1"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.743494 21509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal/instance:
uuid: "03953defb747457f8fb6b42feefb1a54"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "976f1717a15657a17307884fb528981c"
server_key_iv: "b3b2d7d9781b4d7b528ae5d44171b3a1"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.748234 21509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:54:44.751601 21526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:44.753297 21509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:54:44.753494 21509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal
uuid: "03953defb747457f8fb6b42feefb1a54"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "976f1717a15657a17307884fb528981c"
server_key_iv: "b3b2d7d9781b4d7b528ae5d44171b3a1"
server_key_version: "encryptionkey@0"
I20260811 18:54:44.753650 21509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:44.783763 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } attempt: 3
I20260811 18:54:44.784758 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.784422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40268)
0811 18:54:44.784604 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.784673 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:54:44.785573 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.785339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40274)
0811 18:54:44.785460 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.785509 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:54:44.785778 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.253:41459: Network error: Client connection negotiation failed: client connection to 127.27.115.253:41459: connect: Connection refused (error 111)
I20260811 18:54:44.785879 21509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:44.789355 21509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:44.789613 21509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:44.797760 21509 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.253:41459
I20260811 18:54:44.798075 21586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.253:41459 every 8 connection(s)
I20260811 18:54:44.799091 21509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/data/info.pb
I20260811 18:54:44.799820 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21509
I20260811 18:54:44.799988 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-1/wal/instance
I20260811 18:54:44.800315 28111 external_mini_cluster.cc:1428] Setting key bd453d3d8b7c7d8b592da2659f02b236
I20260811 18:54:44.803268 21587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:44.808212 21587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.115.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.115.252@KRBTEST.COM" created.
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:44.825314 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.810963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.27.115.254:42581)
0811 18:54:44.811175 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:44.811182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:44.812184 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 18:54:44.813566 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:44.813570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:44.813641 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:44.813759 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:44.815217 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.815767 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.817031 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.817378 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.817510 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.820858 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:44.820863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:44.820866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:44.820907 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:44.822653 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.823354 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.823359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.823361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.823459 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.823904 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.823907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.823909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.824190 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:44.824318 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:44.824967 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 18:54:44.825121 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 18:54:44.826411 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.811421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42581 (local address 127.0.0.1:59734)
0811 18:54:44.811991 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.812044 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:44.813312 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:44.813967 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:44.813975 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:44.814457 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:44.815037 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.815053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.815929 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.815932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.816870 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.816882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.817594 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.817600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.817781 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.818650 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:44.818670 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:44.820616 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:44.822837 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.822845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.822868 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.823221 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.823569 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.823572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.823575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.823781 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.824313 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:44.824321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:44.824646 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 18:54:44.824898 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 18:54:44.825332 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":147,"threads_started":1}
I20260811 18:54:44.828533 21587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
Entry for principal kudu/127.27.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab.
I20260811 18:54:44.834568 28111 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4WGUPP/build/debug/bin/kudu
/tmp/dist-test-task4WGUPP/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:45479
--webserver_interface=127.27.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.115.212:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:44.841342 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.829262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40284)
0811 18:54:44.829399 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.829412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:44.829497 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:44.830276 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:44.830316 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:44.830576 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:44.830704 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.830711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.831868 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.831871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.832529 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.832540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.833521 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.833525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.833608 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.834372 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:44.834392 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:44.835983 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:44.839076 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.839080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.839082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.839371 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.839814 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.839816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.839818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.839910 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.840764 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:44.840767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:44.840929 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 18:54:44.841052 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:54:44.841193 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 18:54:44.841742 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.829310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.27.115.253:41459)
0811 18:54:44.829700 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:44.829712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:44.829768 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:54:44.829829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:44.829834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:44.830016 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:44.830195 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:44.830861 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.831706 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.832658 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.833378 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.833515 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.836167 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:44.836191 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:44.836208 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:44.836233 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:44.838873 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.839539 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.839551 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.839556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.839694 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.840022 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.840028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.840033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.840515 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:44.840704 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:44.841229 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 18:54:44.841445 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":200,"threads_started":1}
I20260811 18:54:44.844383 21587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:44.845400 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.845050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41084)
0811 18:54:44.845218 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.845316 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 18:54:44.845651 21527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
I20260811 18:54:44.846704 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.846446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41098)
0811 18:54:44.846580 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.846641 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 18:54:44.846884 21587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:45479: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
I20260811 18:54:44.896232 21587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } attempt: 1
I20260811 18:54:44.897202 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.896873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41102)
0811 18:54:44.897041 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.897117 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 18:54:44.897972 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.897747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41106)
0811 18:54:44.897863 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.897916 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 18:54:44.898156 21587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:45479: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
I20260811 18:54:44.929093 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } attempt: 4
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:44.940438 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.929933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40286 (local address 127.27.115.253:41459)
0811 18:54:44.930120 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:44.930153 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:44.930246 (+ 93us) server_negotiation.cc:407] Connection header received
0811 18:54:44.930459 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:44.930463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:44.930524 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:44.930614 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:44.931169 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.932068 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.933369 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.933634 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.933756 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.936395 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:44.936398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:44.936400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:44.936431 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:44.938252 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.938985 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.938989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.938992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.939143 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:44.939480 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:44.939484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:44.939486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:44.939721 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:44.939842 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:44.940132 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:54:44.940263 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:54:44.940438 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.929841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40286)
0811 18:54:44.930112 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.930157 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:44.930316 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:44.930667 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:44.930670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:44.930889 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:44.931009 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.931015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.932206 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.932213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.933234 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:44.933246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:44.933766 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:44.933770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:44.933879 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:44.934563 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:44.934576 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:44.936184 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:44.938418 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.938422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.938426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.938805 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.939263 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:44.939266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:44.939269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:44.939369 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:44.939832 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:44.939835 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:44.939949 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 18:54:44.940070 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 18:54:44.940254 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 18:54:44.941862 21501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:44.942879 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.942483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41116)
0811 18:54:44.942727 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.942790 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 18:54:44.943734 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.943480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41126)
0811 18:54:44.943628 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.943679 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 18:54:44.943982 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:45479: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
I20260811 18:54:44.966327 21587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } attempt: 2
I20260811 18:54:44.967396 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.967095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41132)
0811 18:54:44.967244 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.967319 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 18:54:44.967262 21593 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:54:44.967658 21593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:44.967731 21593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:44.967814 21593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:54:44.968185 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:44.967970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41148)
0811 18:54:44.968078 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:44.968130 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 18:54:44.968323 21587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:45479: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
W20260811 18:54:44.972275 21593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:44.972376 21593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:44.972415 21593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:44.972455 21593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:44.972486 21593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:44.978293 21593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.115.212:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.115.254:42581,127.27.115.253:41459,127.27.115.252:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/krb5kdc/kudu_127.27.115.252.keytab
--principal=kudu/127.27.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.115.252:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:44.979570 21593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:44.980708 21593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:44.989943 21602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.989923 21604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:44.989934 21600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:44.990011 21593 server_base.cc:1034] running on GCE node
I20260811 18:54:44.990912 21593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:44.992218 21593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:44.993453 21593 hybrid_clock.cc:648] HybridClock initialized: now 1786474484993425 us; error 50 us; skew 500 ppm
Aug 11 18:54:44 dist-test-slave-9jjr krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:44.996883 21593 init.cc:377] Logged in from keytab as kudu/127.27.115.252@KRBTEST.COM (short username kudu)
I20260811 18:54:44.998062 21593 webserver.cc:460] Webserver started at http://127.27.115.252:43197/ using document root <none> and password file <none>
I20260811 18:54:44.998723 21593 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:44.998823 21593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:44.999096 21593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:45.001541 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data/instance:
uuid: "839fe31e75e84799be5acee795bbf9ea"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "e7c86ef7843f3fab61cf11169389eb19"
server_key_iv: "c08afbc7154156334f11b2dc0cb294ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:45.002455 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal/instance:
uuid: "839fe31e75e84799be5acee795bbf9ea"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "e7c86ef7843f3fab61cf11169389eb19"
server_key_iv: "c08afbc7154156334f11b2dc0cb294ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:45.003223 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } attempt: 1
I20260811 18:54:45.004297 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.003929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41150)
0811 18:54:45.004116 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.004178 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:54:45.005188 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.004954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41152)
0811 18:54:45.005070 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.005129 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 18:54:45.005329 21501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.115.252:45479: Network error: Client connection negotiation failed: client connection to 127.27.115.252:45479: connect: Connection refused (error 111)
I20260811 18:54:45.007623 21593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:54:45.010998 21611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:45.013195 21593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:54:45.013401 21593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data,/tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal
uuid: "839fe31e75e84799be5acee795bbf9ea"
format_stamp: "Formatted at 2026-08-11 18:54:44 on dist-test-slave-9jjr"
server_key: "e7c86ef7843f3fab61cf11169389eb19"
server_key_iv: "c08afbc7154156334f11b2dc0cb294ae"
server_key_version: "encryptionkey@0"
I20260811 18:54:45.013583 21593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:45.037482 21593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:45.041272 21593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:45.041471 21593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:45.049676 21593 rpc_server.cc:307] RPC server started. Bound to: 127.27.115.252:45479
I20260811 18:54:45.049986 21671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.115.252:45479 every 8 connection(s)
I20260811 18:54:45.051020 21593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/data/info.pb
I20260811 18:54:45.054181 21672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:45.055474 28111 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4WGUPP/build/debug/bin/kudu as pid 21593
I20260811 18:54:45.055670 28111 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4WGUPP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474173680671-28111-0/minicluster-data/master-2/wal/instance
I20260811 18:54:45.055982 28111 external_mini_cluster.cc:1428] Setting key cde244ddae1515814be53b3cb9a3c133
I20260811 18:54:45.059520 21587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } attempt: 3
I20260811 18:54:45.059325 21672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.253@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:45.072485 21501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } attempt: 2
I20260811 18:54:45.072630 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.060122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41164)
0811 18:54:45.060362 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.060382 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:45.060486 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:45.060947 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:45.060950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:45.061170 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:45.061273 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.061279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.063388 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.063395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.063953 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.063963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.065227 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.065339 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.065948 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:45.065964 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:45.067718 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:45.070107 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.070110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.070112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.070445 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.070926 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.070933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.070937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.071040 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.071992 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:45.071996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:45.072114 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 18:54:45.072257 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 18:54:45.072444 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 18:54:45.073549 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.061615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.27.115.254:42581)
0811 18:54:45.061762 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.061766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.062207 (+ 441us) server_negotiation.cc:407] Connection header received
0811 18:54:45.062427 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.062431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.062489 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.062590 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.063453 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.064238 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065314 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.065669 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065734 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.069239 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.069242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.069245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.069273 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.071203 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.071882 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.071886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.071888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.071980 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.072358 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.072361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.072363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.072591 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.072727 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.073263 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.073396 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:54:45.074793 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.061530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.254:42581 (local address 127.0.0.1:59736)
0811 18:54:45.062050 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.062102 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:45.062311 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:45.062663 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:45.062669 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:45.063158 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:45.063318 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.063330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.064398 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.064402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.065161 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.065175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.065985 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.066177 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.066798 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:45.066825 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:45.069064 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:45.071336 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.071344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.071350 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.071778 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.072124 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.072127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.072129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.072244 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.072795 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:45.072802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:45.072982 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 18:54:45.073193 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 18:54:45.073579 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1,"spinlock_wait_cycles":4864,"thread_start_us":126,"threads_started":1}
I20260811 18:54:45.074788 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.060177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.27.115.252:45479)
0811 18:54:45.060662 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.060681 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.060696 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:45.060731 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.060734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.060780 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.060888 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.061444 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.063207 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.064084 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.065070 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065449 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.067882 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.067913 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.067934 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.067968 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.069978 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.070594 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.070600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.070606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.070794 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.071163 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.071165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.071167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.071830 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.072032 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.073902 (+ 1870us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.074230 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":311,"server-negotiator.queue_time_us":364,"spinlock_wait_cycles":4992,"thread_start_us":194,"threads_started":1}
I20260811 18:54:45.074788 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.057619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.27.115.252:45479)
0811 18:54:45.058215 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.058236 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.058283 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:54:45.059257 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.059282 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.059808 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.060220 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.061766 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.063209 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.064508 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.065070 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.065293 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.068623 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.068627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.068630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.068664 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.071676 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.072223 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.072230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.072234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.072325 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.072982 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.072986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.072988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.073217 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.073387 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.074184 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.074290 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":369,"server-negotiator.queue_time_us":408,"thread_start_us":150,"threads_started":1}
I20260811 18:54:45.077181 21672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:45.080132 21587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: Bootstrap starting.
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.254@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
I20260811 18:54:45.083511 21587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:45.084563 21587 log.cc:824] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:45.084770 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.073018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41170)
0811 18:54:45.073316 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.073332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:45.073410 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:45.074614 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:45.074617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:45.074821 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:45.074946 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.074954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.076607 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.076611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.077363 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.077376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.078551 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.078558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.078657 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.079256 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:45.079272 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:45.080884 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:45.082955 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.082959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.082961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.083255 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.083622 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.083625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.083627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.083731 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.084226 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:45.084229 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:45.084333 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 18:54:45.084436 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:54:45.084589 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260811 18:54:45.085343 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.073567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.27.115.252:45479)
0811 18:54:45.074335 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.074352 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.074367 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:45.074404 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.074406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.074454 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.075613 (+ 1159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.075747 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.076473 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.078118 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.078421 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.078500 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.081049 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.081053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.081056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.081084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.082813 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.083398 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.083401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.083403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.083499 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.083868 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.083872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.083873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.084087 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.084951 (+ 864us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.085041 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.085176 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":155,"threads_started":1}
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
I20260811 18:54:45.088675 21587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: No bootstrap required, opened a new log
I20260811 18:54:45.088977 21501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: Bootstrap starting.
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.254@KRBTEST.COM
I20260811 18:54:45.092402 21501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:45.093483 21501 log.cc:824] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:45.093562 21587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.093894 21587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03953defb747457f8fb6b42feefb1a54, State: Initialized, Role: FOLLOWER
I20260811 18:54:45.094394 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.077778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.253:41459 (local address 127.0.0.1:40300)
0811 18:54:45.077963 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.077979 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:45.078064 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:45.082946 (+ 4882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:45.082952 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:45.083156 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:45.083310 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.083319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.084399 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.084402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.085501 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.085513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.086245 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.086248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.086356 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.086888 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:45.086903 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:45.088452 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:45.091442 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.091446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.091449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.091727 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.092991 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.092995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.092997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.093110 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.093735 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:45.093738 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:45.093865 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 18:54:45.093987 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 18:54:45.094208 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 18:54:45.094377 21587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.095018 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.082511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.27.115.253:41459)
0811 18:54:45.082639 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.082644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.082659 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:45.082698 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.082701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.082743 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.082816 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.083469 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.084252 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.085630 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.085953 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.086060 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.088640 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.088645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.088647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.088678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.090895 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.092743 (+ 1848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.092748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.092752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.092863 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.093246 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.093251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.093254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.093565 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.093676 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.094676 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.094862 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:45.095471 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.082342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.27.115.254:42581)
0811 18:54:45.082483 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.082488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.082505 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:54:45.082548 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.082551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.082600 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.082687 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.083809 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.084494 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.086226 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.086551 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.086642 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.089856 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.089860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.089863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.089892 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.092573 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.093197 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.093201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.093202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.093307 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.093735 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.093739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.093740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.094077 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.094302 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.094947 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.095305 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:54:45.095798 21672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:54:45.096859 21587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:45.097656 21683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.097822 21683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.115.252@KRBTEST.COM for kudu/127.27.115.252@KRBTEST.COM
Aug 11 18:54:45 dist-test-slave-9jjr krb5kdc[21414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.115.253@KRBTEST.COM
W20260811 18:54:45.104578 21696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:45.104712 21696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:45.107386 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.096483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.115.252:45479 (local address 127.0.0.1:41186)
0811 18:54:45.096626 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:45.096643 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:45.096730 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:45.097276 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:45.097279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:45.097497 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:45.097638 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.097660 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.099007 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.099014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.099828 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:45.099841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.100445 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.100449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:45.100521 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.101181 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:45.101198 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:45.103228 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:45.105675 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.105679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.105681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.105944 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.106289 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:45.106292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:45.106294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:45.106381 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:45.106847 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:45.106850 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:45.106939 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 18:54:45.107030 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:54:45.107208 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 18:54:45.101424 21509 master_runner.cc:186] Error getting master registration for 127.27.115.254:42581: OK, instance_id { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" instance_seqno: 1786474484535033 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:54:45.107724 21509 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:54:45.108492 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.097106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.27.115.253:41459)
0811 18:54:45.097213 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.097217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.097232 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:54:45.097279 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.097283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.097336 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.097424 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.098265 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.098974 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.100094 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.100440 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.100518 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.104283 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.104287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.104293 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.104337 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.106228 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.106821 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.106825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.106827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.106927 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.107286 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.107289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.107291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.107527 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.107731 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.108173 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.108333 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":41}
I20260811 18:54:45.108528 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.096743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.27.115.252:45479)
0811 18:54:45.097061 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.097066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.097079 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:54:45.097113 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.097117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.097161 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.097772 (+ 611us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.098014 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.098892 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.099974 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.100322 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.100817 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.103426 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.103430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.103432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.103457 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.105519 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.106060 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.106063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.106065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.106187 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.106482 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.106485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.106486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.106741 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.108146 (+ 1405us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.108229 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.108370 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260811 18:54:45.108906 21501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: No bootstrap required, opened a new log
I20260811 18:54:45.112376 21672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: Bootstrap starting.
I20260811 18:54:45.113108 28111 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:54:45.114344 21475 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "302bfdb596ec4002b332cc921a2ccb76"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59738
I20260811 18:54:45.115504 21672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:45.115605 21501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.115886 21501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302bfdb596ec4002b332cc921a2ccb76, State: Initialized, Role: FOLLOWER
I20260811 18:54:45.116272 21501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.116477 21672 log.cc:824] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:45.117031 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.117422 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.117908 21501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:45.118014 21475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:54:45.118309 21475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:45.119107 21672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: No bootstrap required, opened a new log
I20260811 18:54:45.119820 21475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.120884 21475 leader_election.cc:288] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [CANDIDATE]: Term 1 election: Requested vote from peers 03953defb747457f8fb6b42feefb1a54 (127.27.115.253:41459), 839fe31e75e84799be5acee795bbf9ea (127.27.115.252:45479)
I20260811 18:54:45.122226 21556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "302bfdb596ec4002b332cc921a2ccb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "03953defb747457f8fb6b42feefb1a54"
I20260811 18:54:45.122563 21556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:45.124390 21556 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 302bfdb596ec4002b332cc921a2ccb76 in term 1.
I20260811 18:54:45.125512 21445 leader_election.cc:302] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03953defb747457f8fb6b42feefb1a54, 302bfdb596ec4002b332cc921a2ccb76; no voters:
I20260811 18:54:45.125401 21672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.125900 21672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 839fe31e75e84799be5acee795bbf9ea, State: Initialized, Role: FOLLOWER
I20260811 18:54:45.126495 21672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
I20260811 18:54:45.128248 21672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:45.128316 21643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "302bfdb596ec4002b332cc921a2ccb76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "839fe31e75e84799be5acee795bbf9ea"
I20260811 18:54:45.128602 21643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:45.130497 21643 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 302bfdb596ec4002b332cc921a2ccb76 in term 1.
I20260811 18:54:45.130977 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.131121 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.128952 21428 master_runner.cc:186] Error getting master registration for 127.27.115.252:45479: OK, instance_id { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" instance_seqno: 1786474485047291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:54:45.133231 21428 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:54:45.133865 21699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:45.134162 21699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [term 1 LEADER]: Becoming Leader. State: Replica: 302bfdb596ec4002b332cc921a2ccb76, State: Running, Role: LEADER
I20260811 18:54:45.134588 21699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } }
W20260811 18:54:45.135126 21712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:45.135244 21712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:45.136755 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 302bfdb596ec4002b332cc921a2ccb76. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.136898 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:45.144577 21738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:45.145705 21738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:45.145740 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.126259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.27.115.253:41459)
0811 18:54:45.126380 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.126385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.131725 (+ 5340us) server_negotiation.cc:407] Connection header received
0811 18:54:45.132054 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.132058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.132115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.132169 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.136159 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.136944 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.138332 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.138686 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.138787 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.141735 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.141742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.141746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.141780 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.143651 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.144223 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.144227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.144229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.144343 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.144663 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.144667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.144668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.144908 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.145043 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.145429 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.145565 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:54:45.152655 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.127646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.27.115.252:45479)
0811 18:54:45.127769 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.127773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.134445 (+ 6672us) server_negotiation.cc:407] Connection header received
0811 18:54:45.134781 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.134785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.134844 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.134897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.136140 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.136933 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.139374 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.139818 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.139916 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.148259 (+ 8343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.148263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.148266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.148298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.150367 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.150969 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.150973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.150975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.151073 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.151452 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.151456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.151459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.151738 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.151913 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.152314 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.152437 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:54:45.152962 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:45.125053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.27.115.254:42581)
0811 18:54:45.125183 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.125188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.131500 (+ 6312us) server_negotiation.cc:407] Connection header received
0811 18:54:45.133278 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.133282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.133340 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.133395 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.135174 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.136010 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.140123 (+ 4113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:45.140536 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:45.140643 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:45.148227 (+ 7584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:45.148231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:45.148234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:45.148280 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:45.150727 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.151249 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.151255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.151258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.151364 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:45.151674 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:45.151678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:45.151680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:45.151913 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:45.152023 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:45.152674 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 18:54:45.152809 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:54:45.153294 21643 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [term 1 FOLLOWER]: Refusing update from remote peer 302bfdb596ec4002b332cc921a2ccb76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:45.154248 21699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:54:45.156147 21736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:54:45.156250 21736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:54:45.159106 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 302bfdb596ec4002b332cc921a2ccb76. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.159271 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.159993 21556 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [term 1 FOLLOWER]: Refusing update from remote peer 302bfdb596ec4002b332cc921a2ccb76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:54:45.161329 21699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:54:45.166620 21683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 302bfdb596ec4002b332cc921a2ccb76. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.166806 21683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.181736 21735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.181974 21735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:45.183538 21683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.183707 21683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.187973 21740 mvcc.cc:205] Tried to move back new op lower bound from 7317399491178033152 to 7317399491115810816. Current Snapshot: MvccSnapshot[applied={T|T < 7317399491178033152}]
I20260811 18:54:45.189109 21745 mvcc.cc:205] Tried to move back new op lower bound from 7317399491178033152 to 7317399491115810816. Current Snapshot: MvccSnapshot[applied={T|T < 7317399491178033152}]
I20260811 18:54:45.190232 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.190392 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76 [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:45.191350 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "302bfdb596ec4002b332cc921a2ccb76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302bfdb596ec4002b332cc921a2ccb76" member_type: VOTER last_known_addr { host: "127.27.115.254" port: 42581 } } peers { permanent_uuid: "03953defb747457f8fb6b42feefb1a54" member_type: VOTER last_known_addr { host: "127.27.115.253" port: 41459 } } peers { permanent_uuid: "839fe31e75e84799be5acee795bbf9ea" member_type: VOTER last_known_addr { host: "127.27.115.252" port: 45479 } } }
I20260811 18:54:45.191497 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:54:45.193207 21738 catalog_manager.cc:1347] Generated new cluster ID: 183649e2c1f843b2b948d691ce66b59f
I20260811 18:54:45.193285 21738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:45.227272 21738 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:45.228736 21738 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:45.250504 21738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 302bfdb596ec4002b332cc921a2ccb76: Generated new TSK 0
I20260811 18:54:45.251960 21738 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:54:45.289214 21504 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.285840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.27.115.254:42581)
0811 18:54:45.286010 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.286016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.287963 (+ 1947us) server_negotiation.cc:407] Connection header received
0811 18:54:45.288204 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.288208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.288261 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.288318 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.289056 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59768: BlockingRecv error: recv got EOF from 127.0.0.1:59768 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 18:54:45.290849 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.286601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.27.115.253:41459)
0811 18:54:45.286717 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.286721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.289796 (+ 3075us) server_negotiation.cc:407] Connection header received
0811 18:54:45.289982 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.289985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.290037 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.290101 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.290719 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40328: BlockingRecv error: recv got EOF from 127.0.0.1:40328 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:54:45.293947 21680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.287312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.27.115.252:45479)
0811 18:54:45.287433 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.287437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.292263 (+ 4826us) server_negotiation.cc:407] Connection header received
0811 18:54:45.292452 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.292456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.292510 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.292563 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.293809 (+ 1246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41202: BlockingRecv error: recv got EOF from 127.0.0.1:41202 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 18:54:45.299691 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.297399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.27.115.253:41459)
0811 18:54:45.297525 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.297530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.298524 (+ 994us) server_negotiation.cc:407] Connection header received
0811 18:54:45.298782 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.298786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.298874 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.298934 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.299571 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40342: BlockingRecv error: recv got EOF from 127.0.0.1:40342 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 18:54:45.306346 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.303849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.27.115.253:41459)
0811 18:54:45.303984 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.303989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.304908 (+ 919us) server_negotiation.cc:407] Connection header received
0811 18:54:45.305283 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.305288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.305366 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.305454 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.306201 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40348: BlockingRecv error: recv got EOF from 127.0.0.1:40348 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:54:45.313303 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.311315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40352 (local address 127.27.115.253:41459)
0811 18:54:45.311450 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.311455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.312075 (+ 620us) server_negotiation.cc:407] Connection header received
0811 18:54:45.312391 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.312397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.312474 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.312530 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.313148 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40352: BlockingRecv error: recv got EOF from 127.0.0.1:40352 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 18:54:45.323642 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.321116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.27.115.253:41459)
0811 18:54:45.321364 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.321369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.322268 (+ 899us) server_negotiation.cc:407] Connection header received
0811 18:54:45.322550 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.322574 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.322672 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.322740 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.323495 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40364: BlockingRecv error: recv got EOF from 127.0.0.1:40364 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
W20260811 18:54:45.337044 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.334857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.27.115.253:41459)
0811 18:54:45.335001 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.335007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.335858 (+ 851us) server_negotiation.cc:407] Connection header received
0811 18:54:45.336118 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.336122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.336186 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.336251 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.336906 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40370: BlockingRecv error: recv got EOF from 127.0.0.1:40370 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:54:45.358762 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.356254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.27.115.253:41459)
0811 18:54:45.356410 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.356417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.357105 (+ 688us) server_negotiation.cc:407] Connection header received
0811 18:54:45.357379 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.357382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.357448 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.357513 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.358600 (+ 1087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40384: BlockingRecv error: recv got EOF from 127.0.0.1:40384 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 18:54:45.394626 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.393224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.27.115.253:41459)
0811 18:54:45.393433 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.393438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.393605 (+ 167us) server_negotiation.cc:407] Connection header received
0811 18:54:45.393779 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.393789 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.393840 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.393935 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.394488 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40392: BlockingRecv error: recv got EOF from 127.0.0.1:40392 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 18:54:45.462644 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.461050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.27.115.253:41459)
0811 18:54:45.461235 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.461240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.461544 (+ 304us) server_negotiation.cc:407] Connection header received
0811 18:54:45.461649 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.461651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.461705 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.461798 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.462533 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40394: BlockingRecv error: recv got EOF from 127.0.0.1:40394 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 18:54:45.594576 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.593229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.27.115.253:41459)
0811 18:54:45.593405 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.593408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.593518 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:54:45.593740 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.593743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.593804 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.593905 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.594470 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40396: BlockingRecv error: recv got EOF from 127.0.0.1:40396 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 18:54:45.854449 21592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:54:45.853216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.27.115.253:41459)
0811 18:54:45.853450 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:45.853455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:45.853502 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:54:45.853561 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:45.853563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:45.853616 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:45.853695 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:45.854315 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40410: BlockingRecv error: recv got EOF from 127.0.0.1:40410 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:54:46.106208 21696 catalog_manager.cc:1259] Loaded cluster ID: 183649e2c1f843b2b948d691ce66b59f
I20260811 18:54:46.106338 21696 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: loading cluster ID for follower catalog manager: success
I20260811 18:54:46.108105 21696 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: acquiring CA information for follower catalog manager: success
I20260811 18:54:46.109382 21696 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03953defb747457f8fb6b42feefb1a54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:54:46.157891 21736 catalog_manager.cc:1259] Loaded cluster ID: 183649e2c1f843b2b948d691ce66b59f
I20260811 18:54:46.158015 21736 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: loading cluster ID for follower catalog manager: success
I20260811 18:54:46.159951 21736 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: acquiring CA information for follower catalog manager: success
I20260811 18:54:46.161419 21736 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 839fe31e75e84799be5acee795bbf9ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:54:46.375666 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:46.370347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.27.115.253:41459)
0811 18:54:46.370682 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:46.370686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:46.370832 (+ 146us) server_negotiation.cc:407] Connection header received
0811 18:54:46.370943 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:46.370945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:46.371097 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:46.371223 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:46.371720 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.372379 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.373567 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.373898 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.374046 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:46.374533 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:46.375053 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:46.375182 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:46.375333 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 18:54:46.375412 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":125,"threads_started":1}
I20260811 18:54:46.384248 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:46.379830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.27.115.252:45479)
0811 18:54:46.380111 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:46.380114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:46.380202 (+ 88us) server_negotiation.cc:407] Connection header received
0811 18:54:46.380369 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:46.380372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:46.380520 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:46.380648 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:46.380928 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.381463 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.382477 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.382731 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.382830 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:46.383239 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:46.383751 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:46.383877 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:46.384004 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 18:54:46.384101 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
I20260811 18:54:46.392458 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:46.387912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.27.115.253:41459)
0811 18:54:46.388091 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:46.388097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:46.388541 (+ 444us) server_negotiation.cc:407] Connection header received
0811 18:54:46.388717 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:46.388722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:46.388793 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:46.388880 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:46.389224 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.390023 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.391149 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.391434 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.391552 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:46.391886 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:46.391985 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:46.392075 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:46.392203 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 18:54:46.392270 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:54:46.392701 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:46.388402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.27.115.252:45479)
0811 18:54:46.388527 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:46.388531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:46.388707 (+ 176us) server_negotiation.cc:407] Connection header received
0811 18:54:46.388879 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:46.388885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:46.388951 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:46.389026 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:46.389386 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.390176 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.391230 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.391583 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.391694 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:46.392162 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:46.392260 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:46.392333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:46.392489 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 18:54:46.392561 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:54:46.393898 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:46.387916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50868 (local address 127.27.115.254:42581)
0811 18:54:46.388274 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:46.388279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:46.388929 (+ 650us) server_negotiation.cc:407] Connection header received
0811 18:54:46.389236 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:46.389240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:46.389431 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:46.389571 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:54:46.389837 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.390644 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.391921 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:46.392267 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:46.392385 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:46.392868 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:54:46.393312 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:54:46.393416 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:46.393624 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 18:54:46.393701 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":69,"threads_started":1}
I20260811 18:54:46.397810 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21428
I20260811 18:54:46.407985 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21509
I20260811 18:54:46.417052 28111 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4WGUPP/build/debug/bin/kudu with pid 21593
2026-08-11T18:54:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3354 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6848 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (312699 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST